feat(objectql,cli): os migrate 新增 summary count/sum 存量 NULL 回填迁移 (#6063) - #6158
Conversation
…count/sum roll-ups (#6063) PR #6013 (#5749) seeds a roll-up's empty-set value at parent INSERT, which reaches new rows only: a database upgraded in place keeps pre-upgrade parents at NULL, because the recompute that would fix them runs only when one of their children is written. Those rows keep vanishing from `= 0` filters, sorts, GROUP BY and formulas. This adds the one-off, explicit data migration for them. - `backfillSummaryNulls` (packages/objectql/src/summary-backfill.ts): walk each object owning a count/sum roll-up, and recompute every row whose column is stored NULL. A pre-upgrade parent WITH children is NULL too and its correct value is the real aggregate, so `SET col = 0 WHERE col IS NULL` is wrong, not merely coarse. Dry run by default; idempotent; driver-agnostic (values are read and tested in JS, no null predicate pushed down); one row's failure is recorded and the run continues. - min/max/avg are never touched: undefined on an empty set, so a stored null there is the correct reading of "no child rows". The report names them as deliberately skipped. - `summary-aggregate.ts`: SummaryDescriptor, summaryEmptySetValue and the single-descriptor aggregate lifted out of engine.ts unchanged, so the seed, the recompute and the backfill share ONE computation instead of three that agree until one is edited. The descriptor gains `childObject` so the parent-side index is usable on its own. - `os migrate summary-nulls`: thin oclif shell over the migration, following the files-to-references precedent (occupancy gate, --apply/--yes, --object, --max-records, --json). No deployment flag — nothing is gated on this run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019Q7oc7ASjh8yxyS3Yz78We
…mary-null-backfill
…ing it (#6063) `check:query-options-erasure` rejects a NEW file in its baseline — the grandfather list only shrinks. The `as any` came along with the code lifted out of engine.ts and is not needed there: `SummaryAggregateEngine.aggregate` takes the query directly, so the cast is dropped and the baseline records only engine.ts's own count falling 13 -> 12. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019Q7oc7ASjh8yxyS3Yz78We
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
📓 Docs Drift CheckThis PR changes 2 package(s): 31 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:
|
…g it to any (#6063) The slot-lookup rule (#4168/#4251) rejects `const engine: any = getService('objectql')` — the sibling migrate commands are silent only because they are grandfathered by file, and that baseline only shrinks. The command's real requirement is `SummaryBackfillEngine` (the slot contract plus the one member the backfill reads), so name that type: the call site keeps its checking and the lookup is not erased. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019Q7oc7ASjh8yxyS3Yz78We
|
补记:第二道本地闸门也红过一次并已修(正文「命令与真实输出」只记了第一道)
来源是照抄兄弟命令的 修法不是加白名单,而是把这个调用点真正需要的类型写出来:命令需要的是「slot 契约 + 迁移读的那一个成员」,即 import type { SummaryBackfillEngine } from '@objectstack/objectql';
...
const engine: SummaryBackfillEngine = stack.kernel.getService('objectql');
复跑(真实输出): 肢 B(注册接线)在这次重建后复验仍成立: 两道本地闸门红→绿的记录都留在这里,是因为它们各自指向同一条纪律:grandfather 名单不是静音键,新文件该做的是把类型写对,不是把自己加进豁免。 Generated by Claude Code |
|
ACCEPT(执行席 PM 验收,冻结期收尾单) 核过:① 三取舍裁定逐条落地 —— 显式 翻 ready + auto-merge,进队列(rc.5 口径)。known-issue 文案的「待 #6063 迁移」从此有了具体指令名: Generated by Claude Code |
Fixes #6063
前提复核(P1-P3,实读 origin/main)
P1 —— #6013 的口径确如裁决描述。成立。
packages/objectql/src/engine.ts(origin/main,合并前行号):829: function summaryEmptySetValue(fn) { return fn === 'count' || fn === 'sum' ? 0 : null; }—— count/sum 的空集值是 0,min/max/avg 是 null,原文自陈是「the ONE place that list is written down」;4350: private initializeSummaryFields(...)在 insert 路径上按该表播种,文档串4347行原文:「Existing rows are untouched: this is create-time only, so parents already stored withnullstaynulluntil a child write recomputes them.」4410: if (value == null) value = summaryEmptySetValue(desc.fn);—— 重算落到同一张表。summary-rollup.test.ts:286起的roll-up summary seeding on the PARENT insert (#5749)整块,含375行leaves avg/max null — undefined on an empty set。⇒ 裁决描述的「insert 起 count/sum 为 0、min/max/avg 保持 null」逐字成立。
P2 —— 存量 NULL 父行在子写入前不会被任何既有路径补齐。成立。
recomputeSummaries的父行集合只来自子记录:ids由recs/prevs的desc.fkField构成(engine.ts4372起),没有第三个来源;initializeSummaryFields是 create-time only(上引4347)。全仓再无别处写这些列。行为侧也钉住了:本 PR 新测dry run reports the same rows and writes nothing里,直写驱动 store 造出的存量父行在回填前读回仍是null。P3 —— 迁移家族结构如裁决引用。成立。
packages/cli/src/commands/migrate/{value-shapes,files-to-references}.ts;scanValueShapes在@objectstack/objectql,runFilesToReferencesMigration在@objectstack/service-storage;--apply才写、报告函数format*Report由命令壳打印。三条前提全部为真,未改道。
裁决引用(issue #6063,2026-08-07 02:07Z)
os migrate,不开机自动跑 —— 落os migrate summary-nulls,命令壳在packages/cli/src/commands/migrate/summary-nulls.ts(薄),真迁移在 objectql;find/aggregate/update,零逐驱动 SQL,且不下推IS NULL谓词(理由见下)。处置
新增
packages/objectql/src/summary-backfill.ts——backfillSummaryNulls(engine, logger, options)+formatSummaryBackfillReport+summaryBackfillComplete。遍历「拥有 count/sum 汇总列」的对象,keyset 走全部父行(只取id+ 汇总列),对读回值为null的行逐行重算并写回。dry-run 默认;幂等;单行失败记录后继续。packages/objectql/src/summary-aggregate.ts—— 从engine.ts原样搬出的 roll-up 内核:SummaryDescriptor、summaryEmptySetValue、单描述符聚合aggregateSummaryValue,外加派生谓词summaryNullIsBackfillable。descriptor 新增childObject字段(父侧索引此前拿不到子对象名,因为子侧索引就是按它做 key)。packages/cli/src/commands/migrate/summary-nulls.ts—— 照files-to-references的壳:occupancy gate(它改行)、--apply/--yes/--force/--object/--max-records/--json、空元数据拒跑。packages/objectql/src/summary-backfill.test.ts—— 10 例。content/docs/deployment/cli.mdx的数据迁移小节。为什么不写
UPDATE ... SET col = 0 WHERE col IS NULL它不只是粗,它是错的:升级前就有子记录的父行同样是 NULL(从没有人重算过它),正确值是真实聚合而不是 0。写 0 等于把「可见地缺失」换成「自信地错误」,而下一次子写入又会把它悄悄改回去 —— 用户什么都没动,列却变了。
为什么不下推
IS NULLnull 谓词的编译正是各驱动分歧最大的地方(
sql-driver-null-operators.test.ts、sql-driver-out-of-contract-filter-input.test.ts的分歧表)。一条覆盖范围取决于「碰巧跑在哪个驱动上」的迁移,等于会静默漏行的迁移。读回值在 JS 里判== null,是唯一在所有驱动上含义相同的写法;代价见「性能口径」。不记 migration flag
两个兄弟迁移记 flag,是因为那面 flag 是后续不可逆行为的开关。本迁移只修值、不改任何 posture,幂等重跑本身就是验证(报告归零即证),记一面没人读的 flag 反而是噪音。已写进模块与命令的文档串。
测试与反向验证
新测(10 例,均绿)
存量状态是直写驱动 store 造出来的 —— 那正是原地升级库的样子(父行早于播种存在,且从没有子写入访问过它)。走引擎 insert 造反而会被 #6013 播成 0,就没得测了。
反向验证(方向先写死,再运行)
肢 A —— 把回填临时改成便宜路线
computed = 0。第二例是意外收获,也是本单最该被看见的伤害形态:便宜路线写了 0 之后,下一次子写入的重算把列改回 42 —— 用户没动任何东西,列自己变了。同源同算的钉子就是钉这个。已恢复原实现,上述 10 例复绿。
肢 B —— 注册接线。
os migrate列表不含它;注册后含。dist/commands/**/*.jsglob 发现,所以「未注册」= 把编译产物移走):注册后的列表项(真实输出节选):
命令与真实输出
合并
origin/main(⛔ 未 rebase)后重跑,origin/main带进了 #5979 的@objectstack/metadata/errors新依赖边,worktree 补跑了一次pnpm install(与本 PR 无关,记录在此以免下一位读者误判):check:query-options-erasure中途红过一次并已修:搬运时把as any一起带了过来,该闸门拒绝新文件进 baseline(grandfather 列表只缩不增)。修法是把它去掉而不是加白名单 —— 新接口SummaryAggregateEngine.aggregate直接收该 query,本来就不需要断言。baseline 最终只记录engine.ts自己的计数 13 → 12。必答项
1. #6013 口径一致性:回填值 / insert 初值 / 重算值三者同源同算?
是,且是结构性同源,不是「今天恰好相等」。 三个调用点读同一份代码:
ObjectQL.initializeSummaryFields→summaryEmptySetValue(desc.fn);ObjectQL.recomputeSummaries→aggregateSummaryValue(this, desc, parentId, execCtx),该函数内部空集回退同样是summaryEmptySetValue(desc.fn);backfillSummaryNulls→ 同一个aggregateSummaryValue(engine, desc, ...)。descriptor 也同源:回填不自己解析 summary 元数据,而是调
engine.getOwnedSummaryDescriptors(object)读引擎自己的索引(同一份 FK 解析、同一份filter、同一条objectRevision失效规则)。范围收窄谓词summaryNullIsBackfillable(fn)由summaryEmptySetValue(fn) !== null派生,不另列函数名单,所以两者不可能各说各话。行为侧还额外钉了一道:测试
writes the SAME value the engine's own child-write recompute would—— 回填后再触发一次真实子写入重算,断言列不动。肢 A 证明这枚钉子会红。2. #5979 / #5351 / #6014 交叠
未触其面。 逐条:
seedAutonumber把读故障答成return 0—— 对已有 N 行的表重新从 1 发号,自增号与既有行相撞且零日志(#4825 同族,活体) #5979(seedAutonumber的catch { return 0 })—— 已于本 PR 合并origin/main时落地(带进@objectstack/metadata/errors依赖边与engine-autonumber-seed-outage.test.ts)。合并零冲突,两边 hunk 不相邻。buildDriverOptions的 ambient 事务)—— 未触。validation/rule-validator.ts)—— 未触,文件不相交。engine.ts 的 hunk 逐个点名(共 4 处,全部落在 roll-up 机制内部,与上述两单的落点
seedAutonumber/buildDriverOptions不相邻):./summary-aggregate.js的三个符号;SummaryDescriptor+summaryEmptySetValue定义处 → 换成一条指向新文件的说明注释(纯搬运,零语义改动);buildSummaryIndex的 descriptor 字面量:补childObject(值就是本来当索引 key 用的那个变量);getOwnedSummaryDescriptors去掉private;recomputeSummaries内联的聚合块 → 调aggregateSummaryValue。共享 index
packages/objectql/src/index.ts也碰了(导出新迁移面),追加式,不改既有行。之所以不是「新文件里再写一遍聚合」:那正是
scan-value-shapes.ts头注释「Why the predicate is imported」写下的教训 —— 第二份实现只要差一个从句,迁移写进去的值就和引擎下一次重算的值不一致。必答项 1 要求「必须同源」,搬运是唯一能给出代码证据的做法。3. 多租户形态
与
value-shapes/files-to-references完全一致,不新造机制:{ isSystem: true }且不带tenantId。buildDriverOptions(engine.ts)只在execCtx.tenantId !== undefined时才下发租户作用域,所以本迁移在共享库多 org 形态下一次覆盖库内所有 org 的行 —— 这正是部署级回填需要的;--database-url(或OS_DATABASE_URL)指定。命令的 occupancy gate、确认提示与报告都是按「一次一个目标库」设计的;4. 性能口径
父行 × count/sum 汇总列对数),与裁决口径一致。已回填的行第二次跑不再产生任何聚合(幂等),所以量级只随「真实缺口」走,不随库大小走;pageSize500,只取id+ 汇总列)。这是为了不下推IS NULL而付的代价,与scanValueShapes(500/页)、backfillFileReferences(200/页)同形,先例照抄;withTransientRetry(bug(objectql): 汇总重算失败仅 warn——子记录批量插入报成功,父 summary 静默过期且无重试 #3147)。这样一行失败只损失一行,而且重跑只会回访仍是 NULL 的行;--max-records可为每个对象设上限,超限报告truncated,不假装跑完;--object可只跑已知受影响的对象;N NULL row(s)再决定窗口。顺带
无越界发现,未新开 issue。⛔ 未碰
content/docs/releases/(known-issue 文案由发布流程从 issue 取材)。Generated by Claude Code