feat(gfql): verified column-stat facts in the index registry (typed-ontology phase 1) - #1854
Conversation
…ntology phase 1) gfql_index_col_stats() / gfql_index_all(): per-column min/max/null-count facts for the bound node id and edge endpoint columns, same identity+fingerprint validity as physical indexes. The dense two-hop count kernel consumes them conservatively: valid facts proving containment skip the O(E) endpoint-bounds scan; insufficient facts fall back to the scan, never decline. Registry copies via dataclasses.replace so added fields can't be dropped silently. Both-sides pins: skip/scan gate, identity invalidation, conservative-miss fallback, index_all inclusion. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MF7uRZLKZaD6Q9FGWSmyXi
Nodes-role facts also carry n_unique, so an UNFILTERED equal-domain count (domain == the bound node frame) elides the interval scan when the fact proves density; filtered domains always scan (a full-frame fact cannot prove a subset dense). Pinned both ways with count parity. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MF7uRZLKZaD6Q9FGWSmyXi
|
DGX container matrix on the current head ( 🤖 Generated with Claude Code |
…eted API (owner review) Point-by-point on #1854: (2) build_col_stats_fact declines ONLY by explicit precondition (absent col, non-integer, empty); reductions run bare so real errors reraise; null-bearing int cols get null_count with min/max omitted. (3) eager targeting: node_columns/edge_columns params, explicit unfactable requests raise by name, binding defaults skip; laziness deferred to ontology Q7. (4) approximation DIRECTION stated on _facts_prove_bounds + interval hint (under-approximate provability). (5) boundary matrix: precondition declines, propagating reduction errors, null-gate both sides, exact bounds edges, explicit-request raises, gapped-id hint refusal. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MF7uRZLKZaD6Q9FGWSmyXi
|
Review rework pushed — point by point:
🤖 Generated with Claude Code |
|
Container matrix on the rework head ( 🤖 Generated with Claude Code |
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MF7uRZLKZaD6Q9FGWSmyXi
|
Lock-lane receipts are in, and they refute the benefit on the board shape — reporting per the receipts-only rule. Candidate build master+this ( Root cause is structural: this graph is multi-typed, so the full edge frame's endpoint bounds include city/country ids outside the Person interval — the conservative full-frame fact can never prove containment for a rel-filtered subset and always falls back to the scan. Sound by design, but the flagship count shape gets nothing. Facts would need to be per-relationship-type (typed-ontology phase 2) to earn their bind cost here. Per the stated merge condition ('helping & not hurting, locked in pyg-bench'), this does not qualify for auto-merge: not hurting ✓, helping ✗. Options: (a) hold until per-type facts exist and re-lane (my recommendation — this PR then becomes their foundation), (b) merge as infrastructure accepting the +4.4ms indexed-bind cost, (c) close and fold into phase 2. Owner's call. 🤖 Generated with Claude Code |
|
Motivation found and measured (owner's question: any real scenario?). On a Pokec-shaped graph — ONE node type, ONE relation, dense ids, 100k nodes / 2.4M edges, the exact shape of the mgBench suite Memgraph markets on — the facts fire as designed and save 2.48ms = 19.5% of the whole two-hop count (12.74 → 10.25ms, value-identical, exploratory micro-benchmark on the dev box, engine=polars). The board suite's multi-type graph was the one shape that can't benefit; homogeneous graphs — the classic social-network benchmark class — get the full win. Proper receipted Pokec lane available on request before merge. 🤖 Generated with Claude Code |
…branch CHANGELOG keeps both entries; gfql_fast_paths.py auto-merged (the projection work and the fact consult touch different regions of the count path). Verified locally: the affected suites pass apart from the 69 cudf cases that fail IDENTICALLY on origin/master here (libnvrtc.so.12 absent on this box -- #1855's territory), baselined before attributing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MF7uRZLKZaD6Q9FGWSmyXi
CHANGELOG keeps all entries; code and test files auto-merged. Verified locally: only the 3 cudf cases that fail identically on origin/master here (libnvrtc absent) fail; zero non-cudf failures across the lowering and grouped-aggregate suites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MF7uRZLKZaD6Q9FGWSmyXi
Registry keyed (role, column, type_column, type_value); build/consult/ tests/benchmark-motivation to follow. Parked per owner sequencing: benefit-lock and merge #1854/#1856 first. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MF7uRZLKZaD6Q9FGWSmyXi
Registry keyed (role, column, type_column, type_value); build/consult/ tests/benchmark-motivation to follow. Parked per owner sequencing: benefit-lock and merge #1854/#1856 first. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MF7uRZLKZaD6Q9FGWSmyXi
First fact kind from the typed-ontology design (phase 1 needs no policy answers — it mirrors the existing explicit
gfql_index_*surface and the benchmark I-lanes already measure indexed arms honestly, with build cost in disclosedbind_ms).What:
g.gfql_index_col_stats()(and folded intogfql_index_all()) computes per-column min/max/null-count facts for the bound node id and edge endpoint columns, stored in the index registry under the SAME identity+fingerprint validity contract as the physical indexes — a frame rebind is a safe miss, never a stale answer.Consumption (conservative by construction): full-frame bounds contain every row subset's bounds, and zero nulls on the frame means zero nulls on any subset — so the dense two-hop
count(*)kernel skips its O(E) endpoint-bounds scan when valid facts prove containment, and falls back to the scan (never declines) when they cannot. This targets the remaining q8@100k proof cost alongside the landed round-4 projection.Robustness: registry copy methods move from positional construction to
dataclasses.replace, so an added field can never again be silently dropped by a copy method.Tests (both sides at every boundary): fact build + values per engine (pandas/polars/cudf); identity invalidation (equal-valued copy of the frame must MISS); the fact gate both ways (facts ⇒ no scan; no facts ⇒ scan; same count); conservative-miss (an out-of-domain endpoint on a row the rel filter drops makes facts insufficient — must scan AND serve);
gfql_index_allinclusion. Container matrix run reporting below.For owner review — not in the self-merge authorization.
🤖 Generated with Claude Code
https://claude.ai/code/session_01MF7uRZLKZaD6Q9FGWSmyXi