Best FrameX Speedup
Groupby Key Sum (1 worker)
2.307xbaseline / FrameX
Benchmark Report
FrameX vs Native: Performance Dashboard
Snapshot from the local benchmark suite. Use this report to track wins, regressions, and where optimization work should focus next.
Run Performance Test
python3 -m benchmarks.benchmark_suiteHighlights
Slowest FrameX Case
Join Inner (1 worker)
0.633xbaseline / FrameX
Best C Backend Case
Reduction Min Max F64 (1 worker)
0.991xbaseline / FrameX
Performance
Native vs FrameX operation timings.
| Scenario | Workers | native | framex | Speedup | Winner | RSS (native/framex) |
|---|---|---|---|---|---|---|
| Filter Val2 Gt 500 | 1 | 118.7 µs | 76.8 µs | 1.545x | FrameX Side | — / — |
| Groupby Key Sum Mean | 1 | 454.2 µs | 207.2 µs | 2.192x | FrameX Side | — / — |
| Join Inner | 1 | 380.2 µs | 600.4 µs | 0.633x | Baseline Side | — / — |
| Ndarray Np Sin Dispatch | 1 | 197.1 µs | 165.3 µs | 1.192x | FrameX Side | — / — |
| Ndarray Sum | 1 | 6.2 µs | 7.8 µs | 0.800x | Baseline Side | — / — |
| Sort Val1 | 1 | 2.371 ms | 2.422 ms | 0.979x | Baseline Side | — / — |
Parallel Processing
Threaded kernel scaling by worker count.
| Scenario | Workers | native | framex | Speedup | Winner | RSS (native/framex) |
|---|---|---|---|---|---|---|
| Numeric Kernel Threads | 1 | 134.8 µs | 139.8 µs | 0.964x | Baseline Side | — / — |
| Numeric Kernel Threads | 2 | 147.9 µs | 178.6 µs | 0.828x | Baseline Side | — / — |
| Numeric Kernel Threads | 4 | 257.3 µs | 265.0 µs | 0.971x | Baseline Side | — / — |
Single Core
Single-worker baseline overhead and throughput.
| Scenario | Workers | native | framex | Speedup | Winner | RSS (native/framex) |
|---|---|---|---|---|---|---|
| Numeric Kernel | 1 | 90.2 µs | 142.0 µs | 0.635x | Baseline Side | — / — |
Multiprocessing
Process-based object-heavy workload scaling.
| Scenario | Workers | native | framex | Speedup | Winner | RSS (native/framex) |
|---|---|---|---|---|---|---|
| Object Kernel Processes | 1 | 429.862 ms | 422.515 ms | 1.017x | FrameX Side | — / — |
| Object Kernel Processes | 2 | 466.965 ms | 421.897 ms | 1.107x | FrameX Side | — / — |
| Object Kernel Processes | 4 | 488.280 ms | 472.171 ms | 1.034x | FrameX Side | — / — |
Memory
Runtime and peak RSS comparisons.
| Scenario | Workers | native | framex | Speedup | Winner | RSS (native/framex) |
|---|---|---|---|---|---|---|
| Filter Val2 Gt 500 | 1 | 151.8 µs | 93.9 µs | 1.617x | FrameX Side | 168.06 MB / 168.06 MB |
| Groupby Key Sum | 1 | 434.5 µs | 188.4 µs | 2.307x | FrameX Side | 168.56 MB / 168.56 MB |
| Ndarray Np Sin Dispatch | 1 | 198.3 µs | 188.3 µs | 1.053x | FrameX Side | 168.61 MB / 168.59 MB |
C Backend
FrameX Python backend vs C backend kernels.
| Scenario | Workers | python_backend | c_backend | Speedup | Winner | RSS (python_backend/c_backend) |
|---|---|---|---|---|---|---|
| Elementwise Add F64 | 1 | 6.4 µs | 6.7 µs | 0.954x | Baseline Side | — / — |
| Elementwise Scalar Mul F64 | 1 | 8.4 µs | 9.3 µs | 0.901x | Baseline Side | — / — |
| Reduction Mean F64 | 1 | 7.5 µs | 7.6 µs | 0.986x | Baseline Side | — / — |
| Reduction Min Max F64 | 1 | 47.1 µs | 47.5 µs | 0.991x | Baseline Side | — / — |
| Reduction Sum F64 | 1 | 7.4 µs | 7.7 µs | 0.969x | Baseline Side | — / — |
Workload Capability Matrix Check
Automated runtime checks from benchmarks.check_framex_workloads.
| Workload | Status | Runtime | Detail |
|---|---|---|---|
| Single-machine ETL (~100MB–30GB) | PASS | 248.01 ms | ETL parquet round-trip + filter/groupby/sort succeeded |
| Analytics with joins | PASS | 2.16 ms | Join produced 60000 rows as expected |
| ML preprocessing (mixed numeric + categorical) | PASS | 8.21 ms | Mixed numeric/categorical preprocessing flow succeeded |
| Large NumPy operations | PASS | 8.55 ms | NEP13/18 dispatch and chained ndarray ops succeeded |
| Distributed clusters (multi-node) | PARTIAL | 0.15 ms | Neither Ray nor Dask distributed backend is installed |
| GPU acceleration | PARTIAL | 0.08 ms | CuPy not installed; GPU path unavailable in this environment |
| Production streaming | PASS | 0.35 ms | Micro-batch StreamProcessor API succeeded with transform pipeline |