Add TPC-DS SLT tests with plans and Parquet parity

[joseph-isaacs]

Summary

Follow-up to #9794 (ClickBench): the same SLT layout for TPC-DS at scale factor 0.1.

It also carries the DuckDB scan-merging fix that #9803 ended up not landing. Without it the DuckDB TPC-DS suite returns wrong results for q28, which is why CI was red here.

Changes

• slt/tpcds/generate_data.sh generates the 24 tables with DuckDB's tpcds extension through uvx, exports them to Parquet, converts each to Vortex with vx convert, and then runs duckdb/parity.slt, which reads every table in both formats through DuckDB and fails if any row differs. slt/generate_data.sh accepts tpcds like the other datasets; CI already calls the wrapper.
• Per engine: create.slt.no, drop.slt.no, results/q1..q99.slt.no and plans/q1..q99.slt.no from the upstream queries in vortex-bench/sql/tpcds, run unchanged. tpcds.slt asserts results and EXPLAIN plans (EXPLAIN (FORMAT json) with explain_output = 'all' on DuckDB) against Vortex; parquet.slt reruns the same result records against Parquet so both formats must agree. All 99 queries run on both engines with no errors, and no tie-breaking edits to the queries were needed.
• The runner registers TPC-DS as a generated-data suite, so its tests are ignored until both fixture formats exist. README updated.

Fixture size: 14MB, of which 12MB is the DuckDB JSON plans (the format the TPC-H PR chose).

The DuckDB scan-merging fix

DuckDB's common-subplan optimizer keys a scan on its serialized form. The Vortex table function keeps pushed-down filters, projections and aggregates in its FFI bind data, which has no serializer, so two scans of the same file with different pushed-down filters serialized identically and were merged into one shared CTE, handing one branch the other's rows. Parquet is unaffected because its filters live in table_filters, which are serialized. Setting verify_serialization = false is how DuckDB expects a function with opaque bind data to opt out of plan serialization, and therefore of subplan merging.

TPC-DS q28 is the case in point: its six quantity buckets all returned the fifth bucket's average. slt/duckdb/common_subplan_pushdown.slt is a smaller regression test over a single file; without the fix its second branch reports the first branch's average.

The TPC-H q11 DuckDB plan is regenerated as a consequence: its scalar subquery is no longer folded into a shared CTE, so the plan gains the scans it previously read back through CTE_SCAN. The CTE and CTE_SCAN nodes disappear, no cardinality estimate changes value, and q11's results are unchanged.

Checks

• cargo nextest run -p vortex-sqllogictest with TPC-H, TPC-DS and ClickBench fixtures present: TPC-DS passes on both engines, and the new regression test passes.
• The regression test was confirmed to fail with the one-line fix removed (105 455 became 105 105), so it does catch the bug.
• clang-format --dry-run --Werror on the changed C++ file.

One local test, tpch/datafusion/tpch.slt, fails in this working copy only: its plans pin file_groups partition counts, and the local .vortex fixtures predate the onpair 0.2.1 compression change, so the files split into a different number of groups. CI regenerates fixtures from scratch and that test passed there on the previous head.

🤖 Generated with Claude Code

https://claude.ai/code/session_01L57FHoJ15heJVNaqgRq7xB

[codspeed-hq]

Merging this PR will degrade performance by 7.04%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
❌ 2 regressed benchmarks
✅ 2194 untouched benchmarks
⏩ 218 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	Simulation	random_i8[0.5]	67.8 µs	91.4 µs	-25.83%
❌	WallTime	words_gather_scalar_avx2[65536]	8.2 µs	9.4 µs	-12.21%
⚡	Simulation	random_i16[0.95]	95.6 µs	77.5 µs	+23.39%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing claude/tpcds-slt-plans (ab68187) with develop (8e2aa05)}

Footnotes

1. 218 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[joseph-isaacs]

CI status on the previous head (a8ed1a2):

• SQL logic tests failed on the TPC-H DuckDB plan for q11: with Vortex scans no longer merged by DuckDB's common-subplan optimizer (the fix in Fix DuckDB scans over Vortex being merged or stalling #9803), q11's scalar subquery is not folded into a shared CTE any more, so its expected plan changed. Regenerated in Fix DuckDB scans over Vortex being merged or stalling #9803 (385e8ec); this PR is rebased on it (9e66771). Results are unchanged.
• Rust tests (coverage) (tests) failed only because vortex-btrblocks::schemes::integer::tests::compress_large_int hit the 150s nextest timeout under coverage instrumentation. Neither PR touches vortex-btrblocks or anything it depends on, and the same job with the same code on Fix DuckDB scans over Vortex being merged or stalling #9803 (06fa41d) passed its test step. Both jobs re-run on the new heads.

---

Generated by Claude Code