Convert trivial filters into slices during reduction, MaskValues::last uses BitBuffer::last_set_index - #9831
Convert trivial filters into slices during reduction, MaskValues::last uses BitBuffer::last_set_index#9831robert3005 wants to merge 1 commit into
Conversation
Merging this PR will regress 15 benchmarks
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | WallTime | arrow_checked_add_u32_neon[16384] |
12.3 µs | 20.4 µs | -39.88% |
| ❌ | Simulation | random_i8[0.5] |
67.8 µs | 93.1 µs | -27.14% |
| ❌ | Simulation | take_filter_primitive_nullable_slice_mask_random_indices[4096, 1000] |
99.9 µs | 130.9 µs | -23.68% |
| ❌ | Simulation | take_filter_primitive_nullable_slice_mask_random_indices[16384, 1000] |
118.4 µs | 151.1 µs | -21.6% |
| ❌ | Simulation | filter_list_primitive_short[Sparse] |
132.6 µs | 168.3 µs | -21.18% |
| ❌ | Simulation | filter_list_primitive_wide[Sparse] |
133.5 µs | 168.9 µs | -20.94% |
| ❌ | Simulation | filter_list_primitive_short[Clustered] |
134.2 µs | 168.4 µs | -20.3% |
| ❌ | Simulation | filter_list_primitive_short[Prefix] |
137.8 µs | 169.6 µs | -18.74% |
| ❌ | Simulation | filter_list_primitive_wide[Clustered] |
137.9 µs | 169.2 µs | -18.52% |
| ❌ | Simulation | filter_list_primitive_wide[Prefix] |
138.2 µs | 169 µs | -18.22% |
| ❌ | Simulation | filter_powerlaw_by_random[10000] |
32.3 µs | 37.9 µs | -14.83% |
| ❌ | Simulation | take_filter_primitive_slice_mask_sequential_indices[16384, 1000] |
52.8 µs | 61 µs | -13.48% |
| ❌ | WallTime | dict_canonicalize_gt_u8_neon[1000000] |
486.9 µs | 560 µs | -13.05% |
| ❌ | WallTime | words_gather_scalar_avx2[65536] |
8.2 µs | 9.4 µs | -12.24% |
| ❌ | WallTime | dict_canonicalize_gt_u8_neon[16000000] |
8.4 ms | 9.3 ms | -10.07% |
| ⚡ | Simulation | density_sweep_single_slice[0.9999] |
84.2 µs | 27.3 µs | ×3.1 |
| ⚡ | Simulation | density_sweep_single_slice[0.999] |
84 µs | 27.3 µs | ×3.1 |
| ⚡ | Simulation | density_sweep_single_slice[0.99] |
83.7 µs | 27.3 µs | ×3.1 |
| ⚡ | Simulation | density_sweep_single_slice[0.95] |
82.2 µs | 27.3 µs | ×3 |
| ⚡ | Simulation | density_sweep_single_slice[0.9] |
80.4 µs | 27.3 µs | ×2.9 |
| ... | ... | ... | ... | ... | ... |
ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing rk/trivialfilter (d997574) with develop (8e2aa05)
Footnotes
-
218 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
Polar Signals Profiling ResultsLatest Run
Previous Runs (10)
Powered by Polar Signals Cloud |
Benchmarks: PolarSignals Profiling 📖Commits: PR datafusion / vortex-file-compressed / ns (1.016x ➖, 0↑ 1↓)
File Size Changes (1 files changed, +8.6% overall, 1↑ 0↓)
Totals:
|
Benchmarks: TPC-H SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.994x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.000x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.013x ➖, 0↑ 2↓)
duckdb / parquet / ns (1.002x ➖, 0↑ 0↓)
File Size Changes (8 files changed, +16.7% overall, 6↑ 2↓)
Totals:
|
Benchmarks: FineWeb NVMe 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.984x ➖, 3↑ 2↓)
datafusion / parquet / ns (1.008x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.926x ➖, 3↑ 3↓)
duckdb / parquet / ns (1.035x ➖, 0↑ 1↓)
File Size Changes (1 files changed, +25.7% overall, 1↑ 0↓)
Totals:
|
Benchmarks: Clickbench Sorted on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.736x ✅, 10↑ 0↓)
datafusion / parquet / ns (1.006x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.720x ✅, 9↑ 0↓)
duckdb / parquet / ns (0.989x ➖, 0↑ 0↓)
File Size Changes (100 files changed, +26.4% overall, 100↑ 0↓)
Totals:
|
Benchmarks: TPC-DS SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.005x ➖, 0↑ 1↓)
datafusion / parquet / ns (1.005x ➖, 0↑ 1↓)
duckdb / vortex-file-compressed / ns (0.978x ➖, 8↑ 3↓)
duckdb / parquet / ns (0.998x ➖, 2↑ 3↓)
File Size Changes (24 files changed, +3.0% overall, 16↑ 8↓)
Totals:
|
Benchmarks: TPC-H SF=10 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.992x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.002x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.983x ➖, 0↑ 1↓)
duckdb / parquet / ns (0.991x ➖, 0↑ 0↓)
File Size Changes (8 files changed, +17.1% overall, 6↑ 2↓)
Totals:
|
Benchmarks: FineWeb S3 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.247x ➖, 0↑ 3↓)
datafusion / parquet / ns (0.963x ➖, 1↑ 0↓)
duckdb / vortex-file-compressed / ns (1.153x ➖, 0↑ 1↓)
duckdb / parquet / ns (1.007x ➖, 0↑ 0↓)
|
Benchmarks: Statistical and Population Genetics 📖Commits: PR How to read Verdict and Engines
duckdb / vortex-file-compressed / ns (0.860x ✅, 6↑ 0↓)
duckdb / parquet / ns (1.022x ➖, 0↑ 0↓)
File Size Changes (1 files changed, +16.6% overall, 1↑ 0↓)
Totals:
|
Benchmarks: Clickbench on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.893x ✅, 13↑ 1↓)
datafusion / parquet / ns (0.997x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.963x ➖, 10↑ 4↓)
duckdb / parquet / ns (1.006x ➖, 2↑ 0↓)
File Size Changes (100 files changed, +29.3% overall, 100↑ 0↓)
Totals:
|
Benchmarks: TPC-H SF=1 on S3 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.272x ➖, 0↑ 7↓)
datafusion / parquet / ns (1.155x ➖, 0↑ 5↓)
duckdb / vortex-file-compressed / ns (1.173x ➖, 0↑ 6↓)
duckdb / parquet / ns (1.105x ➖, 0↑ 0↓)
|
…t uses BitBuffer:last_set_index Signed-off-by: Robert Kruszewski <github@robertk.io>
156426e to
d997574
Compare
Optimise more filters away