perf(gfql): grouped-aggregate fast path declares its plan columns (round-4 follow-up) - #1856
Conversation
Round-4 follow-up: node/edge filters project to ids + plan-referenced props on every engine (same monotone _filter_project admission rule as the count path). Missing referenced props stay excluded so the existing decline fires unchanged. Pins: narrow widths + decoy exclusion per engine with eager-twin parity; missing-prop decline parity. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MF7uRZLKZaD6Q9FGWSmyXi
|
DGX container matrix on head 🤖 Generated with Claude Code |
Owner review: projection pins now run pandas + polars + polars-gpu + cudf (width/decoy, missing-prop decline), plus a served-shapes x engines value differential with decoy columns vs the same-engine eager twin. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MF7uRZLKZaD6Q9FGWSmyXi
|
Cross-platform container matrix ( 🤖 Generated with Claude Code |
|
Lock-lane receipts (build Target cells (grouped-agg class) vs the r4 base lanes — all within slot noise: q1@100k 5.72x (base 5.81x), q3@100k 3.46x (3.57x), q4@100k 1.45x (1.33x), and 20k likewise. No harm anywhere; no receipted improvement either — expected in hindsight: this suite's node/edge frames are only 3–5 columns wide, so projecting to ids+props trims almost nothing. The change's thesis (wide production frames) is real but unmeasurable on this benchmark. Incidental but useful: q8@20k read 2.15ms on this lane vs 2.51/2.59 on the two sibling lanes over the same base — the r4 overhead is real on the cross-lane median (~+0.5ms) but per-lane noise straddles the 2.35 floor, which sharpens the case for #1857's deterministic fix rather than relying on any single lane. Per the stated merge condition ('non-regression + improvement, locked'): non-regression ✓, improvement ✗ on this suite — same decision shape as #1854: (a) merge as architecture with the no-harm receipts (it simplifies the WHEN story and costs nothing measurable), (b) hold for a wide-frame benchmark that can show the benefit, (c) close. Owner's call; CI is a verified 74-job green either way. 🤖 Generated with Claude Code |
|
Motivation found and measured (owner's question: any real scenario?). On production-width frames — 60 decoy node columns + 10 decoy edge columns (event-table shape), 200k nodes / 1.5M edges — the grouped-agg projection is 2.18× faster end-to-end: 52.51 → 24.11ms vs current master, value-identical (exploratory micro A/B on the dev box, engine=polars, PYTHONPATH-pinned trees). The benchmark suite's 3–5-column frames were the one shape with nothing to trim. Proper receipted wide-variant lane available on request before merge. 🤖 Generated with Claude Code |
The gate lives inside _filter_project, so the newly-projected grouped-aggregate filters inherit it: on polars, frames under the threshold now skip narrowing. That is the declared contract (AT LEAST the projected columns, exactly them only when large), but #1856's width pin asserted exact narrowing unconditionally and tripped on its tiny fixtures. Both sides of that boundary are now pinned per engine: the width pin forces the large-frame arm (_force_narrowing), and a new small-frame twin asserts polars keeps the decoy columns while pandas/cudf still narrow -- and that neither changes the answer. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MF7uRZLKZaD6Q9FGWSmyXi
…branch CHANGELOG keeps both entries; gfql_fast_paths.py auto-merged (the projection work and the fact consult touch different regions of the count path). Verified locally: the affected suites pass apart from the 69 cudf cases that fail IDENTICALLY on origin/master here (libnvrtc.so.12 absent on this box -- #1855's territory), baselined before attributing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MF7uRZLKZaD6Q9FGWSmyXi
Registry keyed (role, column, type_column, type_value); build/consult/ tests/benchmark-motivation to follow. Parked per owner sequencing: benefit-lock and merge #1854/#1856 first. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MF7uRZLKZaD6Q9FGWSmyXi
Cross-engine depth for the typed consult, matching the bar set on #1856: the 3-arm gate (none / whole-frame / per-type) now runs per engine, so a partition fact that resolves on one backend and not another is a failure rather than an unmeasured gap. cudf arms exercise in CI's GPU lane (this box has no libnvrtc, which fails every cudf case here identically on master). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MF7uRZLKZaD6Q9FGWSmyXi
Registry keyed (role, column, type_column, type_value); build/consult/ tests/benchmark-motivation to follow. Parked per owner sequencing: benefit-lock and merge #1854/#1856 first. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MF7uRZLKZaD6Q9FGWSmyXi
Cross-engine depth for the typed consult, matching the bar set on #1856: the 3-arm gate (none / whole-frame / per-type) now runs per engine, so a partition fact that resolves on one backend and not another is a failure rather than an unmeasured gap. cudf arms exercise in CI's GPU lane (this box has no libnvrtc, which fails every cudf case here identically on master). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MF7uRZLKZaD6Q9FGWSmyXi
1 participant
The declared follow-up from #1853: the single-hop grouped-aggregate fast path (graph-benchmark q1/q3/q4 class) passes
project=to its cached filters — ids plus exactly the props the compiled plan references — on every engine, under the same static monotone admission rule (_filter_project). Both its fused polars lane and the eager/pandas tail were verified to read only those columns.A referenced prop missing from the node frame stays excluded from the projection, so the existing missing-prop decline fires exactly as before (pinned: decline parity with the eager twin, including the error case).
Tests: narrow-width + decoy-exclusion pins per engine with eager-twin value parity; missing-prop decline parity; the existing 199-test differential net for this lane stays green.
For owner review.
🤖 Generated with Claude Code
https://claude.ai/code/session_01MF7uRZLKZaD6Q9FGWSmyXi