perf(gfql): small-frame eager gate for projected filters (fix-forward on r4) - #1857
Merged
Conversation
… on r4) The floors gate caught it: the polars lazy filter+select's fixed per-call plan cost regressed q8@20k past its locked floor while the same projection improved q8@100k. Frames under 1M rows now filter eagerly and cut columns via a buffer-share select -- identical output contract, no added overhead; large frames keep the lazy narrow gather. Width/decoy/error pins unchanged and green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MF7uRZLKZaD6Q9FGWSmyXi
…teration) Even eager filter+select re-tripped the q8@20k floor (2.40 vs 2.35): every extra polars op carries fixed overhead a small frame cannot amortize. Below the row threshold the filter now returns full width -- byte-identical to the pre-projection path -- and the contract becomes AT LEAST the projected columns (exactly them once the frame is large enough to pay; pandas/cudf mask+loc stays narrow at every size). Mechanism pins force the threshold to keep asserting narrow shapes on small fixtures. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MF7uRZLKZaD6Q9FGWSmyXi
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MF7uRZLKZaD6Q9FGWSmyXi
The gate lives inside _filter_project, so the newly-projected grouped-aggregate filters inherit it: on polars, frames under the threshold now skip narrowing. That is the declared contract (AT LEAST the projected columns, exactly them only when large), but #1856's width pin asserted exact narrowing unconditionally and tripped on its tiny fixtures. Both sides of that boundary are now pinned per engine: the width pin forces the large-frame arm (_force_narrowing), and a new small-frame twin asserts polars keeps the decoy columns while pandas/cudf still narrow -- and that neither changes the answer. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MF7uRZLKZaD6Q9FGWSmyXi
lmeyerov
added a commit
that referenced
this pull request
Aug 7, 2026
CHANGELOG keeps all entries; code and test files auto-merged. Verified locally: only the 3 cudf cases that fail identically on origin/master here (libnvrtc absent) fail; zero non-cudf failures across the lowering and grouped-aggregate suites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MF7uRZLKZaD6Q9FGWSmyXi
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The #172-pattern floors gate did its job on the fresh r4 board lanes: q8@20k tripped its locked floor (1.88 → 2.51ms vs max 2.35) while q8@100k improved (13.50 → 13.11ms, q5 back to WIN 1.46x, board 8W/0T/1L). Root cause: the polars lazy filter+select carries a fixed per-call plan cost the monotonicity analysis missed — negligible at 13ms cells, decisive at 1.9ms ones.
Fix: frames under 1M rows filter eagerly and cut columns via a buffer-share select (identical output contract, zero added overhead — the pre-r4 cost shape); large frames keep the lazy narrow gather that won at 100k. All projection pins (width, decoy, error parity, all engines) unchanged and green.
The quarantined 20k lane stays unpublished per the landing tool's refusal; a re-lane on master+this validates the fix and restores the q8@20k floor before this merges (per the lock-in discipline).
For owner review; lock lane queued behind the current dgx chain.
🤖 Generated with Claude Code
https://claude.ai/code/session_01MF7uRZLKZaD6Q9FGWSmyXi