refactor(spec,drivers,lint): one implementation of the filter identity reduction

.changeset/shared-filter-verdict-reduction.md

@@ -0,0 +1,45 @@
+---
+"@objectstack/spec": minor
+"@objectstack/driver-sql": patch
+"@objectstack/driver-mongodb": patch
+"@objectstack/driver-memory": patch
+"@objectstack/lint": patch
+---
+
+refactor(spec,drivers,lint): one implementation of the filter identity reduction (#5659)
+
+`{ $and: [] }` matches every row, `{ $or: [] }` matches none, `{}` is a TRUE
+disjunct that absorbs its `$or`, `{ $not: {} }` is FALSE. That is a ruling
+(#5322/#5134) pinned for every backend by the four identity cases in
+`FILTER_LOGIC_CASES` — and it was implemented four times over: `reduceFilterNode`
+in `driver-sql`, the same function again in `driver-mongodb`, the
+`every`/`some`/truthiness algebra of `driver-memory`'s matcher, and nearly a
+fifth hand-written copy inside `@objectstack/lint`, which declined to write one
+and filed this issue instead.
+
+**New in `@objectstack/spec` (`@objectstack/spec/data`): `reduceFilterVerdict`**,
+beside the case table that proves it. It answers `'true' | 'false' | 'clause'`
+for a filter node and never throws on its own; each backend's own refusals — the
+undeclared `$`-combinator and the `undefined` comparand in `driver-sql`, the
+query-level keys and the `$null` comparand in `driver-mongodb` — are passed in as
+`FilterVerdictHooks` and are invoked from exactly the positions they were invoked
+from before. `reduceFilterKeyVerdict` answers the same question for one key, which
+is what both SQL and MongoDB emitters consult while walking a node.
+
+**No behaviour changes in the three drivers.** The move is mechanical: the shared
+algebra replaces each private copy, the refusals stay where they were, and the
+`FILTER_LOGIC_CASES` conformance suites are green on both sides of the change —
+including the SQL-inheriting `driver-sqlite-wasm` and `driver-turso`.
+
+**`@objectstack/lint` gains two warnings it was structurally blind to.** The
+`multi: true` unbounded-bulk-write rule (#5482) asked "does this filter have zero
+keys", so a `delete_record` bounded by `filter: { $and: [] }` or
+`filter: { $or: [{}] }` — a whole-object write by the ruling every driver executes
+— passed silently. It now asks the reduction, and it warns about both while
+staying quiet on `{ $or: [] }` and `{ $not: {} }`, which match nothing. The
+message names the shape it saw (`a filter that REDUCES TO TRUE ({"$and":[]})`)
+rather than calling a non-empty filter "empty".
+
+If you have a flow declaring a bulk write bounded by one of those two shapes, the
+lint will now tell you so — the write was already unbounded at run time; only the
+feedback is new.

packages/drivers/driver-memory/src/memory-matcher.ts

@@ -16,6 +16,11 @@
  * comparand is not `[min, max]` (#5328).
  */
 
+// [#5659] The Filter Protocol's boolean identity reduction, shared with
+// driver-sql, driver-mongodb and the flow linter. See `evaluate` for why a
+// record-at-a-time matcher consults a record-INDEPENDENT verdict first.
+import { reduceFilterVerdict } from '@objectstack/spec/data';
+
 import { assertFilterConditionShape } from './filter-refusal.js';
 
 type RecordType = Record<string, any>;
@@ -41,9 +46,34 @@ export function match(record: RecordType, filter: any): boolean {
     return evaluate(record, filter);
 }
 
+/**
+ * [#5659] The boolean identities come from the SHARED reduction; everything
+ * record-dependent is still decided here.
+ *
+ * This matcher used to answer `{ $and: [] }`, `{ $or: [] }`, `{}` and
+ * `{ $not: {} }` as a by-product of the JS it happens to be written in —
+ * `every` over an empty array is `true`, `some` is `false`, an empty node falls
+ * through to the trailing `return true`. Those four answers are not an accident
+ * of `Array.prototype`, they are a ruling (#5322/#5134) that `driver-sql` and
+ * `driver-mongodb` each spell out in a `reduceFilterNode` of their own, and
+ * "emergent from the evaluator" is not a place a ruling can be read from or
+ * held to. Asking the shared predicate makes this backend's identity answers
+ * the same OBJECT as theirs rather than a third agreeing coincidence.
+ *
+ * It is a pre-pass and not a rewrite of the loops below, because the verdict is
+ * record-independent by construction and the evaluation is not: `'clause'` —
+ * the answer for every filter that carries a real predicate — falls straight
+ * through to the untouched code. And a `'true'` verdict can only be reached by
+ * a filter whose keys are ALL combinators that resolved (a field key always
+ * contributes `'clause'`), so returning early on it skips no field constraint.
+ */
 function evaluate(record: RecordType, filter: any): boolean {
     if (!filter || Object.keys(filter).length === 0) return true;
-
+
+    const verdict = reduceFilterVerdict(filter);
+    if (verdict === 'true') return true;
+    if (verdict === 'false') return false;
+
     // 1. Handle Top-Level Logical Operators ($and, $or, $not)
     // These usually appear at the root or nested.
 

packages/drivers/driver-mongodb/src/mongodb-filter.ts

@@ -26,6 +26,16 @@
 import type { Filter } from 'mongodb';
 import { nextUtcCalendarDay } from '@objectstack/core';
 import { StandardErrorCode } from '@objectstack/spec/api';
+// [#5659] The Filter Protocol's boolean identity reduction, shared with
+// driver-sql, driver-memory and the flow linter and proven against the same
+// `FILTER_LOGIC_CASES` table this driver's conformance suite runs. This file
+// supplies only its own refusals — see `reduceFilterNode` below.
+import {
+  reduceFilterVerdict,
+  reduceFilterKeyVerdict,
+  type FilterVerdict as SharedFilterVerdict,
+  type FilterVerdictHooks,
+} from '@objectstack/spec/data';
 import {
   coerceTemporalValue,
   type TemporalFieldKind,
@@ -64,8 +74,12 @@ function matchNothing(): Filter<any> {
  * - `'true'`  — matches every document; the translator emits no condition.
  * - `'false'` — matches no document; the translator emits {@link matchNothing}.
  * - `'clause'` — carries at least one real predicate; translate it normally.
+ *
+ * [#5659] The vocabulary is `@objectstack/spec`'s now, because the REDUCTION
+ * that produces it is — see {@link reduceFilterNode}. Kept as a local alias so
+ * every use site below still reads `FilterVerdict`.
  */
-type FilterVerdict = 'true' | 'false' | 'clause';
+type FilterVerdict = SharedFilterVerdict;
 
 /**
  * [#5239] Is `value` a Filter Protocol NODE — the shape `FilterConditionSchema`
@@ -135,51 +149,56 @@ function assertFilterNodeList(value: unknown, key: string, path: string): assert
  * emitted document came out empty, is the point. "Nothing was emitted" cannot
  * distinguish "the author wrote an empty group" from "something failed to
  * translate"; a structural verdict has no such blind spot.
+ *
+ * ## [#5659] The algebra is `@objectstack/spec`'s; the REFUSALS are this driver's
+ *
+ * Every paragraph above describes a ruling four consumers had to agree on and
+ * implemented four times — here, in `driver-sql` (whose copy this one was
+ * written to mirror, down to the variable names), in `driver-memory`'s matcher,
+ * and nearly a fifth time inside `@objectstack/lint`. It lives once now, in
+ * {@link reduceFilterVerdict}, proven against the same `FILTER_LOGIC_CASES`
+ * table this driver's conformance suite runs. What stays here is WHICH shapes
+ * this translator refuses and how it words them, handed over as
+ * {@link MONGO_FILTER_VERDICT_HOOKS} and invoked from exactly the positions
+ * they were invoked from before.
  */
 function reduceFilterNode(node: Record<string, unknown>, path: string): FilterVerdict {
-  let sawFalse = false;
-  let sawClause = false;
-  for (const [key, value] of Object.entries(node)) {
-    const verdict = reduceFilterKey(key, value, path);
-    if (verdict === 'false') sawFalse = true;
-    else if (verdict === 'clause') sawClause = true;
-  }
-  return sawFalse ? 'false' : sawClause ? 'clause' : 'true';
+  return reduceFilterVerdict(node, { ...MONGO_FILTER_VERDICT_HOOKS, path });
 }
 
 /** [#5239] The verdict of ONE key of a filter node. */
 function reduceFilterKey(key: string, value: unknown, path: string): FilterVerdict {
-  const here = path ? `${path}.${key}` : key;
-
-  if (key === '$and' || key === '$or') {
-    assertFilterNodeList(value, key, here);
-    let sawTrue = false;
-    let sawFalse = false;
-    let sawClause = false;
-    value.forEach((element, index) => {
-      const elementPath = `${here}[${index}]`;
-      assertFilterNode(element, elementPath);
-      const verdict = reduceFilterNode(element, elementPath);
-      if (verdict === 'true') sawTrue = true;
-      else if (verdict === 'false') sawFalse = true;
-      else sawClause = true;
-    });
-    // `$and: []` → no FALSE, no clause → TRUE (the AND identity).
-    if (key === '$and') return sawFalse ? 'false' : sawClause ? 'clause' : 'true';
-    // `$or: []` → no TRUE, no clause → FALSE (the OR identity). MongoDB itself
-    // answers neither: it rejects the empty array outright
-    // (`$and/$or/$nor must be a nonempty array`), so this filter used to be a
-    // 500-shaped throw rather than a verdict.
-    return sawTrue ? 'true' : sawClause ? 'clause' : 'false';
-  }
+  return reduceFilterKeyVerdict(key, value, { ...MONGO_FILTER_VERDICT_HOOKS, path });
+}
 
-  if (key === '$not') {
-    assertFilterNode(value, here);
-    const inner = reduceFilterNode(value, here);
-    // NOT TRUE ≡ FALSE — so `{ $not: {} }` matches nothing.
-    return inner === 'true' ? 'false' : inner === 'false' ? 'true' : 'clause';
-  }
+/**
+ * [#5659] This translator's half of the reduction: the shape refusals, at the
+ * positions the shared walk visits them.
+ *
+ * The two `assert*` functions are wrapped in arrows rather than passed by
+ * reference because they are TypeScript assertion functions, whose narrowing is
+ * meaningless through a property reference. Nothing else about the call changes.
+ */
+const MONGO_FILTER_VERDICT_HOOKS: FilterVerdictHooks = {
+  assertNodeList: (value, key, path) => assertFilterNodeList(value, key, path),
+  assertNode: (value, path) => assertFilterNode(value, path),
+  classifyKey: (key, value, here) => classifyFilterKey(key, value, here),
+};
 
+/**
+ * [#5239] The verdict of ONE **non-combinator** key — and this translator's gate
+ * on what a field constraint may not be.
+ *
+ * `here` is the already-joined path of the key, exactly as the reduction hands
+ * it over; the three combinator arms this used to open with are the shared
+ * walk's now.
+ *
+ * MongoDB's own answer to an empty combinator is worth keeping written down,
+ * because the reduction is what spares this driver from it: `$and`/`$or` with an
+ * empty array is rejected outright (`$and/$or/$nor must be a nonempty array`),
+ * so `{ $or: [] }` used to be a 500-shaped throw rather than a verdict.
+ */
+function classifyFilterKey(key: string, value: unknown, here: string): FilterVerdict {
   // Query-level keys carry no predicate; the emitter skips them and so does the
   // verdict, so the two never disagree about what this node is worth.
   if (QUERY_LEVEL_KEYS.has(key)) return 'true';

packages/drivers/driver-sql/src/sql-driver.ts

@@ -14,6 +14,17 @@ import { parseAutonumberFormat, renderAutonumber, missingFieldValues, isTenancyD
 // `AggregationNodeSchema.function` actually admits.
 import { AggregationFunction } from '@objectstack/spec/data';
 import { STRUCTURED_JSON_TYPES, FILE_REFERENCE_TYPES, MULTI_OPTION_TYPES, NUMERIC_VALUE_TYPES } from '@objectstack/spec/data';
+// [#5659] The Filter Protocol's boolean identity reduction — `$and: []` is TRUE,
+// `$or: []` is FALSE, `{}` is a TRUE disjunct, `$not: {}` is FALSE. One
+// implementation for all four consumers, proven against the same
+// `FILTER_LOGIC_CASES` table this driver's conformance suite runs; this file
+// supplies only its own refusals. See `reduceFilterNode` below.
+import {
+  reduceFilterVerdict,
+  reduceFilterKeyVerdict,
+  type FilterVerdict as SharedFilterVerdict,
+  type FilterVerdictHooks,
+} from '@objectstack/spec/data';
 // `defaultValue` runtime tokens (#4560). The DDL below asks the SPEC — not a
 // list of its own — which `defaultValue`s are instructions rather than literals,
 // so the engine and this driver can never disagree about what may become a
@@ -992,8 +1003,12 @@ function safeShapePreview(value: unknown): string {
  * - `'true'`  — matches every row; the compiler emits NO clause for it.
  * - `'false'` — matches no row; the compiler emits the dialect FALSE constant.
  * - `'clause'` — carries at least one real predicate; compile it normally.
+ *
+ * [#5659] The vocabulary is `@objectstack/spec`'s now, because the REDUCTION
+ * that produces it is — see {@link reduceFilterNode}. Kept as a local alias so
+ * every use site below still reads `FilterVerdict`.
  */
-type FilterVerdict = 'true' | 'false' | 'clause';
+type FilterVerdict = SharedFilterVerdict;
 
 /**
  * [#5134] Is `value` a Filter Protocol NODE — the shape `FilterConditionSchema`
@@ -1351,52 +1366,59 @@ function assertFilterNodeList(value: unknown, key: string, path: string): assert
  * "empty because the author wrote nothing" from "empty because something failed
  * to compile". A structural verdict has no such blind spot, and it lets the
  * emitter guarantee that every group it opens receives at least one clause.
+ *
+ * ## [#5659] The algebra is `@objectstack/spec`'s; the REFUSALS are this driver's
+ *
+ * Everything above describes a ruling (#5322/#5134) that four consumers had to
+ * agree on and implemented four times — here, in `driver-mongodb`, in
+ * `driver-memory`'s matcher, and nearly a fifth time inside `@objectstack/lint`,
+ * which declined to hand-write it and filed #5659 instead. The reduction now
+ * lives once, in {@link reduceFilterVerdict}, proven against the same
+ * `FILTER_LOGIC_CASES` table this driver's conformance suite runs.
+ *
+ * What stays here is what is genuinely this driver's: WHICH shapes it refuses
+ * and with which message. They are handed to the shared walk as
+ * {@link SQL_FILTER_VERDICT_HOOKS} and are invoked from exactly the positions
+ * they were invoked from before, so no wording, code or status moved — the
+ * conformance case-set is green on both sides of the change.
  */
 function reduceFilterNode(node: Record<string, unknown>, path: string): FilterVerdict {
-  let sawFalse = false;
-  let sawClause = false;
-  for (const [key, value] of Object.entries(node)) {
-    const verdict = reduceFilterKey(key, value, path);
-    if (verdict === 'false') sawFalse = true;
-    else if (verdict === 'clause') sawClause = true;
-  }
-  // AND over the node's keys: FALSE dominates, then a real predicate, else TRUE.
-  return sawFalse ? 'false' : sawClause ? 'clause' : 'true';
+  return reduceFilterVerdict(node, { ...SQL_FILTER_VERDICT_HOOKS, path });
 }
 
 /** [#5134] The verdict of ONE key of a filter node. */
 function reduceFilterKey(key: string, value: unknown, path: string): FilterVerdict {
-  const here = path ? `${path}.${key}` : key;
-
-  if (key === '$and' || key === '$or') {
-    assertFilterNodeList(value, key, here);
-    let sawTrue = false;
-    let sawFalse = false;
-    let sawClause = false;
-    value.forEach((element, index) => {
-      const elementPath = `${here}[${index}]`;
-      assertFilterNode(element, elementPath);
-      const verdict = reduceFilterNode(element, elementPath);
-      if (verdict === 'true') sawTrue = true;
-      else if (verdict === 'false') sawFalse = true;
-      else sawClause = true;
-    });
-    // `$and: []` → no FALSE, no clause → TRUE (the AND identity).
-    if (key === '$and') return sawFalse ? 'false' : sawClause ? 'clause' : 'true';
-    // `$or: []` → no TRUE, no clause → FALSE (the OR identity). This is the
-    // half the old compile got backwards: it answered the whole table.
-    return sawTrue ? 'true' : sawClause ? 'clause' : 'false';
-  }
+  return reduceFilterKeyVerdict(key, value, { ...SQL_FILTER_VERDICT_HOOKS, path });
+}
 
-  if (key === '$not') {
-    assertFilterNode(value, here);
-    const inner = reduceFilterNode(value, here);
-    // NOT TRUE ≡ FALSE — so `{ $not: {} }` matches nothing.
-    return inner === 'true' ? 'false' : inner === 'false' ? 'true' : 'clause';
-  }
+/**
+ * [#5659] This driver's half of the reduction: the shape refusals, at the
+ * positions the shared walk visits them.
+ *
+ * `assertFilterNodeList` / `assertFilterNode` are wrapped in arrows rather than
+ * passed by reference because they are TypeScript assertion functions, whose
+ * narrowing is meaningless — and whose declaration requirements are a nuisance
+ * — through a property reference. Nothing else about the call changes.
+ */
+const SQL_FILTER_VERDICT_HOOKS: FilterVerdictHooks = {
+  assertNodeList: (value, key, path) => assertFilterNodeList(value, key, path),
+  assertNode: (value, path) => assertFilterNode(value, path),
+  classifyKey: (key, value, here) => classifyFilterKey(key, value, here),
+};
 
+/**
+ * [#5134] The verdict of ONE **non-combinator** key — and this driver's gate on
+ * everything a field constraint may not be.
+ *
+ * `here` is the already-joined path of the key, exactly as the reduction hands
+ * it over; the three combinator arms this used to open with are the shared
+ * walk's now, and the refusals below are unchanged from when they sat under
+ * them.
+ */
+function classifyFilterKey(key: string, value: unknown, here: string): FilterVerdict {
   // [#5348] Everything still `$`-prefixed at this point is an UNDECLARED
-  // combinator — the three declared ones each returned above. Refused here and
+  // combinator — the shared walk resolved the three declared ones before this
+  // key ever reached the hook (#5659). Refused here and
   // not in the emitter for exactly the reason the two lines below are here, and
   // the reason #5327 gave for `{ field: {} }`: this walk is exhaustive and does
   // not short-circuit, while the emitter is skipped wholesale by a boolean