Skip to content

fix(reporting): restore late-account materializer progress - #1206

Merged
bokelley merged 1 commit into
mainfrom
conductor/fix-reporting-late-account-progress
Sep 25, 2026
Merged

bokelley merged 1 commit into
mainfrom
conductor/fix-reporting-late-account-progress

Conversation

@bokelley

Copy link
Copy Markdown
Contributor

When one reporting account remains continuously due, the materializer can repeatedly select it against its own previous served_at value. A newly enrolled due account stays behind the continuation cursor and receives no materialization until the first account has a due gap.

After an account turn commits, the PostgreSQL materializer now clears its sampling cursor so the next bounded sample starts from the durable scheduling order. Lock misses still advance through a busy prefix. The reset occurs after the transaction exits successfully, preserving rollback behavior, advisory/account lock ordering, and fencing. This addresses continuous-first-account starvation; it does not claim starvation persists after a due gap.

This is the separate PY-RUNTIME-001 / RLX-RT-001 review leaf. The production change is seven lines in reporting/materializer/pg.py; the remaining changes add store and public-process regressions and include the public fixture in installed tests. Total: six files, +1247/-1. Producer publication time and live revocation classification remain separate children.

Identity Exact value
Head dba15b6b0dd8063d38c2df4196491ae64d9720ae
Tree 8154ac8fc0ba2a55c10aed4d7de466a5b0aea142
Sole parent / review base b203bbcff51cb648c1fd605224fb74e5a3465a8f on conductor/fix-reporting-scope-selector-default

Validation from retained originals:

  • Development red/green and mutation: exact-parent store controls failed for both lexical orders while the first account stayed due (2 failed / 2 passed); the corrected implementation passed all four. Restoring the old successful-turn cursor produced both targeted failures again. The later lock/commit-failure/two-worker controls passed. These development executions precede the frozen commit and retain their own source identities and original setup failures.
  • Actual public parent red and development green: the installed b203 parent, with the corrected typed r3 fixture, left EUR without materialization after 58 typed status reads while the first account remained due. Four corrected development cells passed both lexical orders and notification modes, with late-account readiness of 0.498 / 11.070 / 1.850 / 5.586 seconds under the unchanged 45-second watchdog.
  • Fresh frozen installed public proof: the selected dba15 VCS and sdist-built packages each passed 1,371 tests on CPython 3.10.21 with real PostgreSQL, zero failures and zero skips. Each deselected one checkout-only generator-idempotence test, covered by source/generated gates. Across those two packages, all eight late-account/restart cases used separate MCP HTTP processes, PostgreSQL with pool size 1, typed onboarding and exact reads, autonomous workers, independent content digests, accepted receipts, and definitive reconciliation. Both accounts' revision, materialization, digest and receipt identities survived same-artifact restart. Historical source observations isolate scheduling; producer and PostgreSQL clocks are real. This is not acquisition-time finality acceptance.
  • Current-artifact version/floor controls: eight VCS/sdist cells across Python 3.10–3.13 passed 752 tests each (6,016 total; one checkout-only deselection per cell). Four separate CPython 3.10.21 signing/model cells passed 138 each (552 total) at Pydantic 2.13.0/MCP 2.0.0 and current 2.13.5/2.2.0. Their 32 cross-signing and 32 raw-body HTTP controls use TS rc.42 / Node 22.12, not selected rc.45 or test(interop): qualify Python and TypeScript reporting artifacts across stable and skew #1199 qualification.
  • Separate mandatory historical continuation lane: original production_rolling passed all eight installed Python 3.10 B23 (50e35f0a) / hardening (a09878f6) × current VCS/sdist × notification-mode cases. Actual historical page-one/binding/pending state survives current-child activation, SIGKILL and restart; old projectors are fenced after activation. This does not prove universal old-decoder support. The retained child probes establish Python 3.10 minor and installed module origins; full per-disposable-process executable hashes and historical dependency inventories were not retained. The separate B24 continuation four-pass record is not added to these eight.

The complete gate history remains explicit:

  • The original CPython 3.10.21 functional supervisor remains exit 1, 17 passing groups / 2 failing groups: receipt 6/8 and feed 7/9. Those required red executions were within their unchanged deadlines and are not relabeled as an original 19/19 pass.
  • Separate complete runs on CPython 3.12.14 passed receipt 8/8 and feed 9/9, preserving every original parameter, notification-disabled/enabled and before/after arm, real clocks, 60-second child limits, and 1,800/2,700-second group ceilings. They match the unchanged workflows' historical-artifact runtime and qualify only those named configured-runtime lanes.
  • Historical B22 declared Python >=3.10. Its bare datetime.fromisoformat rejects the captured five-fractional-digit PostgreSQL value 2026-09-22T19:50:38.81632+00:00 on actual 3.10.21; 3.12.14 accepts it losslessly. Stable post-failure boundary selection strongly links this mechanism to diagnostic r2, but is not a capture of the child-local argument. Other original values, inner failures and historical dependencies remain unknown. The old declared-support limitation remains open: the 3.12 passes neither waive it nor establish a shared cause for all four original failures or a new materializer source regression.
  • Static 15/16 retains inherited dispatch B110 on dba15. Approved fe1a1cbd / fix(decisioning): preserve sanitized validation fallback #1204 corrects it only in a separate sibling; no suppression or retroactive green. Final aggregate hooks remain required.
  • Original missing-helper, direct evidence-reader/direct-url setup, earlier fixture, and diagnostic-helper failures remain at their original exits. Corrected later executions do not overwrite them. Standalone candidate metadata predates reviewed main fix(types): restore targeting overlay input compatibility #1190; its dependency metadata and exact integrated artifact floor remain integration gates. No Pydantic 2.12 acceptance is claimed.

Selected frozen installed artifacts (candidate metadata adcp==8.0.0b15, not the published beta15 package):

Route Wheel SHA-256 Source archive SHA-256
VCS 70f1fcd4108b3e11d61f4f5a7b190ea91e8248d09e5de65f6dad2e3fa6c4626a 979a6200fb9271dbe6ad64bf40224f68e5b0da5145f8f75685d924a64787f93c
sdist 356e9d9ae32f313fd84b0202fc224677bb2ae29d3a66d386645213c76c3747a6 6bddb9fc04959da7a9981acf5291ec1a32f007984e4afa178c0c43e19fda3d28

The two wheel archives have distinct hashes; their 7,813 member paths and bytes match. Both build-input manifests contain 7,820 entries and hash to f7e5deda5aa8713f6a37625e7c21e289a496488ef98a9e985f6a0dbf13b35d27.

Existing original record references under .context/py-runtime-001-20260922/:

  • store-red-basis.json, red-continuous-late-store/, green-continuous-late-store/, mutation-old-success-cursor-r2/, focused-lock-rollback-workers-r2/.
  • parent-public-red-basis.json, red-parent-typed-public-late-r3/, public-r3-result.json, installed-public-progress-summary.json; raw installed streams in acceptance/production-installed-pg-{vcs,sdist}/production-evidence/.
  • completed-local-gate-disposition.json, completed-installed-runtime-version-disposition.json, completed-installed-floor-signing-disposition.json, runtime/artifact-equivalence.json.
  • configured-python312-rolling-gates-20260922/completed-configured-runtime-gates.json, the two groups' record.json and original.log; acceptance/production-rolling/record.json and acceptance/b24-continuation/record.json retain their separate executions.

Source publication is for independent review only. This draft leaf targets the frozen wire branch; scheduled leaf checks are reported as observed, with no full-CI inference or transfer from #1203. Delivery requires later approved ancestry-preserving integration and fresh aggregate/main checks. Runtime review, dependent finality publication, #1199 qualification, security and release decisions remain separate; this PR grants no merge or package-release approval.

Reset the materializer sampling continuation only after a committed account
turn moves its durable served_at rank. Keep continuation across lock misses
so a busy prefix cannot block later eligible accounts.

Cover continuous producer activity, both account orders, size-one pools,
more than two busy pages, concurrent workers, commit rollback and fencing.
Exercise typed public enrollment, real HTTP/PG delivery, 503-row exact
reads, receipts, reconciliation and restart under continuing catch-up load.
Include the store regression in installed production distributions.
Base automatically changed from conductor/fix-reporting-scope-selector-default to main September 25, 2026 12:40
@bokelley
bokelley merged commit 34cc4b4 into main Sep 25, 2026
4 checks passed
@bokelley
bokelley deleted the conductor/fix-reporting-late-account-progress branch September 25, 2026 12:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant