[3/5] docs: handoff for the EQL v3 type checker design#424
Open
freshtonic wants to merge 17 commits into
Open
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Captures the EQL v3 domain model, three impact maps over eql-mapper (type system, SQL surface, declarations/macros), and the open design questions, so the design session starts fresh with full context. Records one corrected finding: literals are inference sinks (value.rs:19 unifies them with a fresh tvar, not Native), so a token type on EqlValue buys no literal checking. Two subagents disagreed on this; resolved by reading the code. Stable-Commit-Id: q-2r73hgg9ex95aba238a8fbe4kp
freshtonic
force-pushed
the
queue/eql-v3/upgrade-deps
branch
from
July 21, 2026 13:06
479744c to
86a8731
Compare
freshtonic
force-pushed
the
queue/eql-v3/typecheck
branch
from
July 21, 2026 13:06
d3087a7 to
8a24e1a
Compare
Captures the shared understanding from the grill-with-docs session on extending the EQL Mapper type checker for EQL v3. CONTEXT.md: refresh the glossary to the v3 target vocabulary — EqlValue now carries domain identity; EqlTrait drops Contain and is documented as coarse; add term-extraction function, functional-index rewrite, query operand, token type, and domain identity. The old 'known model gap' note becomes 'design decisions in flight' pointing at the ADRs. ADRs (new packages/eql-mapper/docs/adr/): - 0001 functional-index rewrite (term functions, not native operators) — forced by Supabase's lack of CREATE OPERATOR; term-fn selection is the capability check. - 0002 token type as inert identity, not a checked dimension — it buys no safety (encrypted cols never unify; literals are inference sinks). Assessed against EQL/eql-bindings 3.0.2 (released 2026-07-20): additive query-twin bindings only; none of the design decisions change. Stable-Commit-Id: q-06b9k91e7pnntayq3rs11tqznz
…rands Two latent bugs in the trait-bound machinery, harmless only while bound checking is dead code (every encrypted column currently gets EqlTraits::all()). Fix them before the EQL v3 loader makes bounds reachable. - EqlTraits::difference was implemented as XOR (symmetric difference), so UnsatisfiedBounds would list traits the type *has* alongside the ones it is missing. It is now a true set difference (self AND NOT other). - Type::must_implement passed its operands reversed relative to Unifier::satisfy_bounds (implemented.difference(required) instead of required.difference(implemented)). Harmless only because XOR is commutative; wrong the moment difference is corrected. Now consistent. Adds unit tests pinning the asymmetric set-difference semantics and that must_implement reports exactly the missing bounds. Refs CIP-3595. Stable-Commit-Id: q-0e9kbc03hrqef307fg7qb5sqj1
Groundwork for the EQL v3 type checker (ADR-0002). Introduces the token type and its inert per-column domain identity, threading it through the type carriers without making it a checked dimension of unification. - New `TokenType` (integer/text/timestamp/…) and `DomainIdentity` (token type + v3 domain typname, e.g. eql_v3_text_ord_ore). - `EqlValue` widens to (TableColumn, Option<DomainIdentity>, EqlTraits); `ColumnKind::Eql`, `SchemaTableColumn` and the new `Column::eql_with_identity` carry the identity. `Column::eql` stays as a back-compat constructor that defaults the identity to None. - The identity is `None` everywhere for now: the branch still emits the v2 surface and no loader populates it yet (CIP-3598 does). It rides through unification and the associated-type machinery untouched, exactly because it is never inspected there. - ColumnKind loses `Copy` (DomainIdentity is not Copy); the handful of by-value uses now clone. - Also fixes the never-invoked, latently-broken no-bounds arm of the concrete_ty!/EQL() macro (wrong EqlValue arity, missing .into()). eql-mapper suite 79 passing; workspace checks clean; fmt + clippy clean. Refs CIP-3597. Stable-Commit-Id: q-5dxadzsggt5seabqhg3sdj8ht5
Implements the SchemaManager side of the v3 type checker (ADR-0002): per-column capability becomes *observed* from the Postgres domain name instead of a hardcoded EqlTraits::all(). - New schema/eql_domains.rs inverts the eql-bindings v3 catalog (v3::all()) into `typname -> (TokenType, EqlTraits)`. Because the catalog is generated from the same source as the installed SQL domains, the mapping cannot drift from them. Term -> capability: hm->Eq, op/ob->Ord, bf->TokenMatch, empty->storage-only; the JSON SteVec domains (term_json_keys == None) map to JsonLike (provisional, see note). - load_schema now resolves each column's v3 domain identity and traits and builds the column via Column::eql_with_identity. The legacy eql_v2_encrypted composite-type arm is retained for reading pre-v3 schemas; unrecognised domains load as Native. - select_table_schemas.sql now also selects information_schema domain_name: v3 encrypted columns are jsonb-backed DOMAINs, so udt_name reports the base type (jsonb) and the domain typname is only available via domain_name (cf. CIP-3441). - Adds eql-bindings 3.0.1 as a workspace dependency, used only by the proxy's schema loader; eql-mapper stays wire-format-agnostic. Re-exports DomainIdentity / TokenType from eql-mapper's crate root. Unit-tested offline against the catalog (12 tests asserting the SEM-term table, incl. the text hm+ord exception and query-twin exclusion). End-to-end validation needs a database with EQL v3 installed and is deferred; the encrypt-config cross-check is a follow-up within CIP-3598. Refs CIP-3598. Stable-Commit-Id: q-6p9ykbew5b6sjaf5p7y9qza4ft
freshtonic
force-pushed
the
queue/eql-v3/typecheck
branch
from
July 21, 2026 21:27
2058d16 to
e558280
Compare
…+ Contain
Applies three confirmed design decisions on the EQL v3 type checker.
1. Domain identity is non-optional (strict ADR-0002). EqlValue is now
(TableColumn, DomainIdentity, EqlTraits) and ColumnKind::Eql carries a
mandatory DomainIdentity — no more Option. The schema loader always
supplies the real identity; there is no honest v3 identity for a legacy
eql_v2_encrypted column, so the loader now drops the v2 arm and logs a
warning instead of fabricating one (v2 is already retired on this build).
- New helpers: TokenType::{as_domain_str, from_domain_name},
DomainIdentity::{from_domain_name, canonical}, and a test-only
EqlValue::with_canonical_identity.
- The schema!/concrete_ty!/test-helper macros synthesise a canonical
text-token identity by default; schema! also gains an
EQL("<domain>") form to pin an explicit v3 domain (token + OPE/ORE
variant) for tests that care.
2. JSON domains map to JsonLike + Contain, verified against the installed
v3 SQL (cipherstash-encrypt.sql): -> / ->> and @> / <@ are real
operators on eql_v3_json_search (not raise-stubs). Corrects the earlier
JsonLike-only guess.
3. The encrypt-config cross-check is dropped: the domain name is the sole
authority for a column's capability. (ADR-0002 doc update to follow.)
The Contain trait is therefore retained (scoped to JSON), NOT deleted as
the handoff/glossary/ticket assumed — @>/<@ raise only on scalar encrypted
columns. Docs/tickets updated separately.
eql-mapper 79 passing; proxy eql_domains 12 passing; workspace check,
fmt, clippy all clean.
Refs CIP-3597, CIP-3598.
Stable-Commit-Id: q-31sxsrtyrptajjpj0n383cb690
Two design corrections found during implementation:
- Containment is NOT removed in v3. Verified against the installed
cipherstash-encrypt.sql: @>/<@ raise on scalar encrypted columns but are
real, supported operators on encrypted JSON (eql_v3_json_search). The
Contain trait is retained as a JSON-only capability. Fixes the glossary
("removed in v3") and adds a consequence to ADR-0001.
- ADR-0002 amended: the domain name is the sole authority for a column's
capability (no encrypt-config cross-check), and the domain identity is
non-optional (legacy eql_v2_encrypted columns are dropped with a warning
rather than given a fabricated identity).
Refs CIP-3597, CIP-3598, CIP-3599.
Stable-Commit-Id: q-4rd3p778qhjfc5dessqeeg1b2e
…ation
Starts the v3 functional-index rewrite (ADR-0001) by pinning the pipeline
design and landing its hardest, most correctness-sensitive piece: choosing
the eql_v3 term-extraction function for an operator on a given column.
- ADR-0003 records the v3 rewrite pipeline: the stored-value vs query-operand
context split (column domain vs query twin cast targets), operator ->
term-function rewriting, JSON containment retained (not deleted), and the
rule inventory. All operand/term details verified against the installed
cipherstash-encrypt.sql, not the docs.
- DomainIdentity gains stores_hm/op/ob/bf and eq_term_fn/ord_term_fn/
match_term_fn/query_twin. These encode the verified selection rules:
* eq_term only where hm is stored; =/<> otherwise fall back to the ordering
term (an ord-only scalar like integer_ord compares via ord_term, exactly
as eql_v3.eq does).
* ord_term (op) vs ord_term_ore (ob) chosen from the domain typname.
* text is the hm exception (text_ord* stores hm alongside its order term).
* query_twin names the term-only operand domain (eql_v3.query_<bare>).
Unit-tested against the SEM-term table. The rewrite rule that consumes these
lands next.
Refs CIP-3599.
Stable-Commit-Id: q-4fed4c4vg8sehf76wwrn5yaysf
…ex form
First working slice of the v3 rewrite pipeline (ADR-0003). Scalar
comparisons on encrypted columns now emit the functional-index form, and
all cast targets move from the opaque eql_v2_encrypted to real v3 domains.
- New RewriteEqlComparisonOps: `col <op> x` -> `eql_v3.<term>(col) <op>
eql_v3.<term>(x)`, the term function chosen from the column's domain
identity (eq_term / ord_term / ord_term_ore). A domain with no term for
the operator is a capability error.
- Cast rules retargeted (helpers::cast_to_v3_domain): a query operand casts
to the eql_v3.query_* twin, a stored value (INSERT/UPDATE) to the
public.eql_v3_* column domain. Role is detected from the AST context
(post-order traversal + full NodePath ancestry): a value under a
comparison predicate is an operand, otherwise stored.
- CastLiteralsAsEncrypted gains node_types so it can name the target domain;
EqlTerm gains eql_value(); helpers gain eql_v3_term_call / is_comparison_op.
Example:
WHERE salary > $1
-> WHERE eql_v3.ord_term(salary) > eql_v3.ord_term($1::JSONB::eql_v3.query_text_ord)
Transitional: JSON (->, @>, jsonb_*) and aggregate (min/max) rewriting still
emit the eql_v2.* function wrappers over v3-cast operands; those wrappers are
retargeted in the next slices. Tests updated to the current output; the
params fixture gains `EQL: Eq` since a bare (capability-less) v3 column is
storage-only and rightly rejects `=`.
eql-mapper 85 passing; workspace check, clippy, fmt clean.
Refs CIP-3599.
Stable-Commit-Id: q-62kqdp86gcbb66tvwhy69tgn5s
Continues the v3 rewrite (ADR-0003). min/max and the jsonb_* passthrough functions on encrypted columns now rewrite to their eql_v3 counterparts. - RewriteStandardSqlFnsOnEqlTypes resolves a pg_catalog function to its eql_v3 counterpart via the new get_eql_v3_function_name, and rewrites only when one is declared. This fixes the long-standing count bug: count has no eql_v3 counterpart (it runs natively on the encrypted value), so it is no longer rewritten to a non-existent eql_v3.count. - sql_decls: the min/max/jsonb_path_*/jsonb_array_* target declarations move from eql_v2 to eql_v3. The containment declarations (jsonb_array, jsonb_contains, jsonb_contained_by) stay under eql_v2 until the containment slice. min(salary) -> eql_v3.min(salary) jsonb_path_query(col, '$.x') -> eql_v3.jsonb_path_query(col, '$.x'::...) count(col) -> count(col) (unchanged) eql-mapper 85 passing; workspace check, clippy, fmt clean. Refs CIP-3599. Stable-Commit-Id: q-092hd8jb1d31ck4va4n7r2gzet
Continues the v3 rewrite (ADR-0003). `@>`/`<@` on encrypted JSON columns now rewrite to the eql_v3 containment functions. Containment is retained in v3, scoped to JSON (ADR-0002 amendment) — `@>`/`<@` on scalar encrypted columns still raise. - RewriteContainmentOps emits eql_v3.jsonb_contains / jsonb_contained_by. - sql_decls: the jsonb_array / jsonb_contains / jsonb_contained_by target declarations move from eql_v2 to eql_v3. notes @> needle -> eql_v3.jsonb_contains(notes, needle) needle <@ notes -> eql_v3.jsonb_contained_by(needle, notes) No eql_v2.* function names remain in the mapper's rewrite output. eql-mapper 85 passing; clippy, fmt clean. Refs CIP-3599. Stable-Commit-Id: q-6dne3hbwwfsj7q42yqf4da8rqv
LIKE/ILIKE on an encrypted column previously only unified the result with Native, so it bypassed the trait system entirely — a column with no match capability silently accepted LIKE. They now apply the `~~`/`~~*` (and negated) operator rules, so the encrypted LHS must implement TokenMatch and the pattern takes its Tokenized associated type. Adds tests: LIKE on a TokenMatch column type-checks; LIKE on an Eq-only encrypted column is a capability error. (The v3 rewrite of LIKE/@@ to `eql_v3.match_term(a) @> eql_v3.match_term(b)` is a separate follow-up; this fixes the type-checking gap.) eql-mapper 87 passing; clippy, fmt clean. Refs CIP-3599. Stable-Commit-Id: q-429ag1vxhrefpb1vy71sbkzg34
Continues the v3 rewrite (ADR-0003). LIKE/ILIKE on encrypted columns now
rewrite to the fuzzy-match functional form.
- New RewriteEqlMatchOps: `col LIKE pat` -> `eql_v3.match_term(col) @>
eql_v3.match_term(pat)`; `NOT LIKE` wraps it in `NOT (...)`. Fuzzy match
compares bloom-filter terms with @> (containment), so the operator becomes
@> between the two match_term calls, mirroring eql_v3.matches. A column
whose domain stores no bloom filter has no match_term -> capability error.
- Operand role detection (helpers::is_query_operand) now also treats a value
under a LIKE/ILIKE predicate as a query operand, so the pattern casts to the
query twin.
email LIKE 'a%'
-> eql_v3.match_term(email) @> eql_v3.match_term('<ct>'::JSONB::eql_v3.query_text_match)
`@@` (BinaryOperator::AtAt) is the remaining fuzzy-match surface; it needs the
operator declaration to parse first (separate slice).
eql-mapper 88 passing; clippy, fmt clean.
Refs CIP-3599.
Stable-Commit-Id: q-0gdjjy9gdt34jne9yhvs1kjw9m
freshtonic
force-pushed
the
queue/eql-v3/typecheck
branch
from
July 22, 2026 05:51
5e9bd60 to
64b7811
Compare
Continues the v3 rewrite (ADR-0003). `->`/`->>` on encrypted JSON columns are functionalised, since managed Postgres (Supabase) forbids the CREATE OPERATOR DDL the native operators would need (ADR-0001). - RewriteContainmentOps (now the general encrypted-JSON binary-op rule) also rewrites `col -> sel` -> `eql_v3."->"(col, sel)` and `col ->> sel` -> `eql_v3."->>"(col, sel)`. Operator-symbol function names are quoted; ordinary names are not. - CastLiteralsAsEncrypted passes a JSON field selector (an EqlTerm::JsonAccessor, the RHS of ->/->>) as encrypted *text* rather than a jsonb-domain cast, to match eql_v3."->"(json_search, text). notes -> 'medications' -> eql_v3."->"(notes, '<encrypted-selector>') ASSUMPTION TO CONFIRM (flagged in code): the encrypted selector replacement is the selector string itself. If the encrypt pipeline instead produces a jsonb payload for JSON selectors, the selector must be extracted rather than emitted verbatim — a one-line change once confirmed. eql-mapper 88 passing; clippy, fmt clean. Refs CIP-3599. Stable-Commit-Id: q-03j8aeq7we1c74jhkrgrny3rt4
Completes the fuzzy-match surface (ADR-0003). `@@` now parses in the
operator DSL and rewrites like LIKE/ILIKE.
- eql-mapper-macros: the binary-operator parser accepts `@@` (it previously
only accepted `@>` after `@`), so `@@` can be declared.
- sql_decls: declare `<T>(T @@ <T as TokenMatch>::Tokenized) -> Native where
T: TokenMatch` — `@@` requires the match capability.
- RewriteEqlMatchOps now also handles `BinaryOp::AtAt`:
col @@ pat -> eql_v3.match_term(col) @> eql_v3.match_term(pat)
and `is_query_operand` treats an `@@` operand as a query operand so the
pattern casts to the query twin.
eql-mapper 89 passing; eql-mapper-macros 3 passing; clippy, fmt clean.
Refs CIP-3599.
Stable-Commit-Id: q-7s39kd5jdyhfjh1qyhkg2s858f
Validates the schema! EQL("<domain>") explicit-domain form end to end: a
block-ORE ordering column (eql_v3_integer_ord_ore) rewrites through
ord_term_ore (not ord_term) with the query_integer_ord_ore twin. This
exercises the ord-vs-ord_ore distinction that the inert domain identity
(ADR-0002) exists to carry.
seq > 5 -> eql_v3.ord_term_ore(seq) > eql_v3.ord_term_ore(...::eql_v3.query_integer_ord_ore)
eql-mapper 90 passing.
Refs CIP-3599, CIP-3600.
Stable-Commit-Id: q-0pjttkst7g5xwva2ag41v1nk6j
Cleanup pass (CIP-3601), now that the v3 rewrite is in place. - eql-mapper-macros: the EqlTrait / operator parse-error messages listed only Eq, Ord, TokenMatch, JsonLike but Contain is a valid keyword (retained in v3 for JSON containment). Both messages now include Contain. - eql-mapper CONTEXT.md: the "migration in flight" note claimed the code still emits the v2 surface; it now emits v3 (no eql_v2.* names remain in output), with end-to-end validation still pending. Note on the default()/all() reconciliation (the other CIP-3601 item): no production change is needed. schema_delta's collect_ddl never fabricates EQL traits (new/added columns are Native), so its EqlTraits::default() only appears in test assertions, consistent with the schema! macro's bare `EQL` (a capability-less, storage-only v3 column). The former mismatch was the loader's hardcoded all(), which CIP-3598 replaced with domain-derived capability. eql-mapper 90 passing; eql-mapper-macros 3 passing; fmt clean. Refs CIP-3601. Stable-Commit-Id: q-56sd3v5nxbdh5b4kpepyc1mm31
freshtonic
changed the title
freshtonic
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📚 eql-v3 PR · 3 of 5
Part of a queue. The PRs merge in FIFO order — the numbered order below, #1 first. Merging one supersedes the PRs after it until the author runs
git queue sync(rebases the rest onto the merged base) andgit queue submit(retargets their PRs).⏳🟢 #422
queue/eql-v3/setup-skills→main⏳🟢 #423
queue/eql-v3/upgrade-deps→queue/eql-v3/setup-skills⏳🟢 #424
queue/eql-v3/typecheck→queue/eql-v3/upgrade-deps👈 this PR⏳🟢 #426
queue/eql-v3/showcase→queue/eql-v3/typecheck⏳🟢 #427
queue/eql-v3/integration→queue/eql-v3/showcase✅ approved · ♻️ changes requested · ⏳ review pending | 🟣 merged · 🟢 open · ⚫ closed — status as of the last
git queue submit.🥞 Managed by git-queue — do not edit this list by hand.
About this queue
Migrates CipherStash Proxy from EQL v2 to EQL v3 (cipherstash-client 0.34.1-alpha.4 → 0.42.0, EQL 2.3.0-pre.3 → 3.0.2), replacing the opaque eql_v2_encrypted composite type with 53 typed jsonb domains that encode both scalar type and searchable capability in the column type itself.
About this branch
Type checker for v3 domains — extends EQL Mapper to emit
eql_v3_<token>_<capability>domains instead of eql_v2_encrypted(designed, not yet built; the step that makes the migration actually work).