Optimize interleave boolean gather and add comprehensive benchmarks

[joseph-isaacs]

Summary

This PR optimizes the boolean gather path in InterleaveArray and adds comprehensive benchmarks comparing Vortex's implementation against arrow-rs across multiple access patterns.

Changes

Benchmark Suite (vortex-array/benches/interleave.rs)

• Restructured benchmarks to test three distinct access patterns: random (fully random gather), round_robin (structured merge pattern), and single_branch (degenerate gather to one branch)
• Added parametrization across branch counts (N=2, N=4) and nullability (nullable/non-nullable)
• Introduced Combo struct to organize benchmark configurations and ensure Vortex and arrow-rs test identical data
• Added arrow benchmark function to measure arrow-rs interleave kernel performance alongside Vortex
• Unified input generation via gen_inputs() seeded by combo to ensure byte-identical data for fair comparison

Boolean Gather Optimization (vortex-array/src/arrays/interleave/execute/bool.rs)

• Optimized the random-access gather loop by hoisting raw (byte_ptr, bit_offset) pairs per value buffer
• Replaced BitBuffer::value() calls with direct get_bit_unchecked() to avoid redundant bounds checks and struct indexing overhead
• Materialized validity masks into full-length BitBuffer objects per branch to enable uniform unchecked reads instead of per-row Option/Mask dispatch
• Added detailed comments explaining the optimization rationale and safety invariants
• Maintained correctness by validating all bounds once upfront before the tight unchecked loops

Test Coverage (vortex-array/src/arrays/interleave/mod.rs)

• Added interleave_binary_spans_word_boundary() test for two-value gather with out-of-order routing across 64-bit packing word boundaries
• Added interleave_binary_nullable_spans_word_boundary() test for the same scenario with nullability to exercise validity packing

Rationale

The boolean gather is a critical path for InterleaveArray execution. The optimization targets the per-bit read overhead in random-access patterns where consecutive outputs read unrelated source words—eliminating the wide struct index and redundant bounds assertion in BitBuffer::value(). The benchmark suite enables direct performance comparison with arrow-rs and validates that the optimization maintains correctness across diverse access patterns and nullability configurations.

Testing

• Existing interleave_bool and interleave_bool_nullable tests continue to pass
• Added two new unit tests covering word-boundary spanning scenarios with and without nullability
• Benchmark suite validates correctness by comparing Vortex and arrow-rs outputs across all Combo configurations
• All changes are covered by existing and new unit tests; no manual testing required

https://claude.ai/code/session_01FmZKDRJtXNGyuSxq5sqFtE

[codspeed-hq]

Merging this PR will degrade performance by 23.61%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

❌ 4 regressed benchmarks
✅ 1524 untouched benchmarks
🆕 6 new benchmarks
⏩ 4 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	Simulation	chunked_bool_canonical_into[(1000, 10)]	20.2 µs	35.6 µs	-43.06%
❌	Simulation	chunked_varbinview_canonical_into[(1000, 10)]	162 µs	198.2 µs	-18.3%
❌	Simulation	chunked_varbinview_into_canonical[(1000, 10)]	177.2 µs	213.2 µs	-16.89%
❌	Simulation	bitwise_not_vortex_buffer_mut[128]	215.3 ns	244.4 ns	-11.93%
🆕	Simulation	vortex[random/n2/nonnull]	N/A	147 µs	N/A
🆕	Simulation	vortex[random/n2/null]	N/A	214.7 µs	N/A
🆕	Simulation	vortex[round_robin/n2/nonnull]	N/A	152.8 µs	N/A
🆕	Simulation	vortex[round_robin/n2/null]	N/A	214.3 µs	N/A
🆕	Simulation	vortex[random/n64/nonnull]	N/A	206.1 µs	N/A
🆕	Simulation	vortex[random/n64/null]	N/A	376 µs	N/A

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing claude/eager-heisenberg-fxi8wx (b582134) with develop (3d7bbfb)}

Footnotes

1. 4 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩