Skip to content

feat(reporting): persist authorized frozen reporting feeds - #1189

Closed
bokelley wants to merge 6 commits into
mainfrom
conductor/reporting-frozen-account-feed-b23
Closed

bokelley wants to merge 6 commits into
mainfrom
conductor/reporting-frozen-account-feed-b23

Conversation

@bokelley

@bokelley bokelley commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

get_reporting_status(view="periods") gains a durable, authorized walk of Core reporting records and the caller's terminal materializations and receipts. Initial capture retains exact historical inputs, releases the account lock and connection before projection, then stores immutable wire records. Every continuation restores those bytes and reauthorizes its caller. Feed-enabled and receipt-only instances can coexist on MCP and A2A.

Refs #1167. B2.3 source integration only. Epoch-0 queues, quarantine and the new-store advertisement veto remain intact. B2.4 activation, aggregate artifact/release acceptance, #1199 and TypeScript acceptance are not supplied by this PR. #1172 remains separately scoped.

Current integrated identity

  • Exact head a9bc445446cb4579f21a73ded9cb71509d83d45f, tree 039f0559870b55df6eb5d3a274207cb364225495.
  • Sole parent composition 8fb84eb5dad9594f30b8df76c46452f3a3fbd687, ordered parents [50e35f0ae3540f19b40e8fc460f5870dfe018bf9, 09fd87f79a746665d828dea66b3a1dd9d1fc189e] (feature, integrated main). Ordinary merge plus appended correction; no history rewrite.
  • Original feature delta 74b338d8..50e35f0a and current PR each contain the same 38 paths, identical in both directions. Full PR: 38 files, +6345/-31.
  • Corrective child: 14 files, +174/-89. git diff --binary --abbrev=8 8fb84eb5..a9bc4454 SHA256 29d60491a24ee9d9a9cf2966a1a33778e8b360d00d781ecdce753bcfe27fe371.
  • Full 09fd87f7..a9bc4454 patch SHA256 5a8ac55a39bdde56535d87031fb372bd7ebfc60938a372a8f079f1ea3c435eaf.
  • No corrective production source, SQL or schema-cache edits. All 464 ledger, 249 status, 10 selector, 187 materializer and 102 receipt manifest entries remain byte-identical to main. The 33 feed entries remain byte-identical to the original feature; no fingerprints regenerated.

Behavior and retained boundaries

The wire allowlist is obligations, revisions, adjustments, caller consumer-status, terminal materializations and revision/adjustment receipts. Private bindings, delivery attempts, pending checks and destinations stay private. Visibility precedes maxima, dependency closure, totals and pagination. Core and caller records retain their committed sequence domains; referenced revisions, obligation ownership, history and receipt dependencies remain available below incremental checkpoints.

Compact rpf1 positions bind account, canonical consumer, semantic filters, both after/through boundaries, snapshot identity, representation and last global key. Every page carries the same checkpoint, advanced by consumers only after exhaustion. Page size/context can vary; semantic filters cannot. Missing or incompatible positions require full replay. Version-1 captured inputs and immutable pages are never rebuilt from current configuration, readability, lifecycle or receipt leaves. Concurrent writers are deferred to another walk, including with a one-connection pool. Nested ledger transactions are rejected at this public boundary.

Receipt ingestion/ordinals/final replay and materializer reserve/I/O/finish retain their transaction boundaries. Feed capture and memory rollback retain all collections, initialized heads and captured parent boundaries. Instance discovery governs actual tools and skills; middleware cannot reuse another call's authorization or frozen projection. Financial receipt numeric parsing stays strict while feed context/extensions allow finite fractional JSON values; raw page limits remain exact bounded integers.

Integration corrections

  1. Catalog fixture: integrated parents expose 763 objects, not the old 742. The test now derives an exact parent mapping from unchanged manifests, asserts exactly ten waiver-binding objects and 763 parents, preserves all original objects, and requires exact equality to parents plus the 33 feed objects. This strengthens the old count-only expectation without tolerance, filtering away failures or changing DDL. The fully enabled catalog contains 1,045 objects.
  2. Version and locale fixtures: mounted/request probe literals and pinned MCP schema tests now use rc.6. The production version guard is unchanged. Installed PG fixtures use the integrated URL/driver helper; three C-only PostgreSQL initialization flags are removed. The catalog must work under actual database collation.
  3. Operational assertions: 42 selected assertions across eleven files now retain operations under -O. Thirty-five extract one first/unconditional outer await (nested awaits stay inside it); seven extract the entire original predicate, preserving multi-await order, preceding operands and short-circuit behavior. An adjacent inverse transform reproduces every original assertion AST at this exact head. No embedded subprocess source literals were changed. Pure snapshot/catalog reads and cancellation joins inside pytest.raises remain intact. Assertions-enabled evaluation order is preserved; assertion-based preconditions under -O are not promised.
  4. Two SQL damage literal pairs are grouped with parentheses only; AST/constants and the six-case damage selection are unchanged by that grouping.
  5. Frozen pins advance to integrated A/B/C/B1/B2.1/B2.2; the exact original beta15/records/integration pins remain.

CI composition and dispatch

The existing required name Postgres conformance tests (Postgres 16) retains always() and permissions: {}. It requires all eight upstream job keys and checks every result for exact success: core/process matrix, status, materializer, native receipts, receipt compatibility, native feed, feed compatibility and installed feed. Failure, cancellation, skipped or empty results fail the aggregate.

20 job keys expand to 25 jobs. Existing lane budgets are unchanged. Original feature feed budgets remain native 25m/20m test, compatibility 50m/45m test with a 2m fetch, and installed 25m/20m test. Each feed job has contents: read; every job has an explicit timeout. The native feed selector uses a shell array built from the whole feed glob, excluding rolling, packaging and installed-PG files before pytest receives explicit paths. nullglob plus an empty-array exit 1 prevents accidental full-tree discovery; quoted array expansion and bash pipefail preserve paths and exit status.

Actual workflow-command collection with PostgreSQL URL present and drivers available confirms a disjoint partition and exact union:

Lane Collected identities
Core 1979
Process 7
Status 251
Materializer 47
Native receipts 637
Receipt compatibility 8
Native feed 375
Feed compatibility 9
Installed feed 6
Full reference selection 3319

Collection executes no test bodies and supplies no execution credit. The actual empty selector exits 1 before pytest. CI/IPR/reviewer provenance is to be authenticated separately. The title workflow retains the original feature's extra parent-branch filter; it is not claimed byte-identical to main.

Local validation and preserved negatives

All results below are fresh implementer-local observations, not CI or independent-review credit:

  • Unchanged composition negative: the migration case failed on 763 != 742; mounted tool inventory failed four cases with VERSION_UNSUPPORTED for rc.3 against rc.6 (four optional PostgreSQL skips). Both original failed records remain intact.
  • Exact head base feed selection: 166 passed,224 intended skips, two credential-construction deprecation warnings; 84.07s pytest. Optional PostgreSQL/frozen/installed coverage is not inferred from skips.
  • Exact head native feed on PostgreSQL16.14, libc en_US.utf8:375 passed,zero skipped/failed/errors; JUnit 958.177s, outer 977.007s under the unchanged1200s local ceiling. Raw stdout SHA256 88fcc6f7d0247e4ac2290d3b8a9ce9a5ccf4edfeb1bbbef56bdcea16aefb482f. Original warnings remain in the raw record.
  • Fresh strict catalog creation/validation under C, libc en_US and ICU en-US:1,045/1,045 objects, exact full mappings, no mismatches in each regime. Every task-owned cluster stopped and removed with no remaining clients.
  • Actual command collections and the empty-selector negative are separate from execution, as recorded above.

Normal Ruff, Black, mypy and adopter hooks passed without bypass. A first hook attempt failed while uv provisioned a newly selected SQLAlchemy 2.1.0 sdist with duplicate normalized extra metadata, before mypy ran. Reusing the existing qualified ignored lock restored the already qualified local environment; no tracked dependency constraint or source was changed. Fresh CI dependency installation remains its own gate. A subsequent Ruff import-order edit was committed through the normal hooks. Private recorder stops are tooling records, not SDK or CI results.

Complete installed Python 3.10 VCS/sdist and the nine real frozen-artifact rolling controls were not run locally for this integrated head. Fresh complete CI must supply those controls and whole-job lifecycle qualification; local pytest ceilings do not qualify CI jobs. Historical original-feature reviews/runs are preserved as historical attribution, never reused as current-head results. The previous 65,461-byte PR body is retained unchanged in the local evidence record.

Security and acceptance limits

Fresh Gitleaks 8.30.1 scans this exact head's 38 full files (749,241 bytes) and all six base-to-head merge-parent commits (146,105,964 added bytes). All input bytes and commit sets were independently reconciled. Full files: zero matches. History: 82 retained matches, individually recomputed from exact source objects (68 schema idempotency examples, 13 patterned documentation credentials at a reserved example endpoint, one verified file digest); zero unadjudicated/private-credential matches. Fresh large-positive and negative controls passed. No repository ignore/baseline, inline allowance or input-size cap; builtin rule allowlists/entropy and pattern-detection limits remain disclosed. Disposition SHA256 bbfa539a55d220778c16dac7f4409ebe3695c210a0d7b2d975b74c5affb83ae7.

The user's standing train policy permits a separately recorded per-head substitution only if the genuine GitGuardian App46505 output explicitly size-skips. This scan is not a GitGuardian success. Fresh actual CodeQL output/annotations and ordinary review/CI gates remain required. The same inherited documented wildcard listener/optional-auth exposure remains disclosed under that standing policy; no alert dismissal, zero-alert claim or exception for new findings is implied.

Six rolling exclusions remain explicit release-note obligations: pre-17ee407a A, pre-0f34c666 B, pre-967b6e28 C, pre-5487f2bd B1, pre-3fd62121 B2.1 and pre-09fd87f7 B2.2 are unqualified. A whole-trigger startup after C remains unsupported; later subset readiness and draining old workers remain required. Historical discovery-path performance risk 9rLB gets no #1191 memoization credit. The inherited legacy fail-closed cursor limitation is not retroactively repaired or given a retention promise by this opt-in feed. Epoch-0/quarantine/advertisement veto, #1199/TS and release holds remain. Guard1201 stays draft LAST; legacy workflow remains disabled.

Comment thread src/adcp/reporting/receipts/handler.py
Comment thread src/adcp/reporting/feed/pg.py Outdated

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve — additive frozen reporting feed feature; no blocking findings.

What I checked

  • New adcp.reporting.feed package is purely additive public surface with correct feat(reporting) semver — no removed/renamed/retyped public exports, no required→optional flips, no enum/discriminator changes.
  • Multi-tenant isolation, HMAC token integrity, and no over-advertisement verified in the projection/snapshot/store wiring.
  • high_risk: true fires only on (modified) files under src/adcp/server/** (a2a_server, idempotency/store, mcp_tools, serve). Per the change-kind rule, modifications with no medium-or-higher finding on those specific files are presumed contract-preserving; the two mediums surfaced are on handler.py and pg.py, not the server files, so row 5 does not fire.
  • No credential-in-ctx_metadata, no generated-code hand-edits, no CI-gate disabling, no import-layering violations.

Medium findings (non-blocking, 2 total → below the row-8 threshold of 3)

  • src/adcp/reporting/receipts/handler.py:187 — broad except Exception flattens legacy LedgerConflictError codes into retryable REPORTING_FEED_STORAGE_UNAVAILABLE.
  • src/adcp/reporting/feed/pg.py:144 — per-account write lock held across full projection with no timeout.

Decision trace: Rows 1 (no critical/high), 2 (gated_paths=false), 3 (no deletions), 4 (mediums are not data-loss/schema/infra category), 5 (mediums not on the modified high-risk server files), 6 (no prior decision), 7 (no team gate), 8 (only 2 mediums) — none fire. Falls through to row 9 → approve.

Worth a follow-up on the two mediums, but neither blocks.

aao-secretariat[bot]
aao-secretariat Bot previously approved these changes Sep 18, 2026

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve — clean delta pass with no blocking or medium findings.

Checked:

  • No no-auto-approve team match; gated_paths is false and review_decision is APPROVED, so no hard gate applies.
  • high_risk is true only via (modified) entries under src/adcp/server/** (a2a_server.py, idempotency/store.py, mcp_tools.py, serve.py). No deletions, and the reviewer surfaced no medium-or-higher concerns on these modified sensitive files, so the modifications are presumed to preserve their contracts. No escalation warranted on the flag alone.
  • The frozen-feed read rework (capture under account lock in one short transaction, release lock+connection, project off-loop via asyncio.to_thread, reauthorize before publication, insert self-contained immutable snapshot in a second lock-free transaction) is correct and well tested: fail-closed caller-transaction guard, reauthorize gating with pg/memory parity, no FK to mutable rows plus random rpfs_ snapshot ids, and legacy-status error redaction keeping provider secrets off the wire.
  • Prior decision was approve with no findings; this run also has none — net stable.

Decision table: rows 1–8 do not fire (no critical/high/medium findings, gate satisfied, no deletions, no team gate, fewer than three mediums). Falls through to row 9 → approve.

Base automatically changed from conductor/reporting-receipt-ingress-b22 to main September 24, 2026 18:17
@bokelley
bokelley dismissed aao-secretariat[bot]’s stale review September 24, 2026 18:17

The base branch was changed.

Comment thread tests/conformance/reporting/test_reporting_feed_transactions.py
Comment thread tests/conformance/reporting/test_reporting_feed_transactions.py
from adcp.types import Error, GetReportingStatusRequest, SyncReportingReceiptsRequest

if TYPE_CHECKING:
from adcp.reporting.feed.store import ReportingFeedStore
Comment thread src/adcp/reporting/feed/store.py
Comment thread src/adcp/reporting/feed/store.py
Comment thread src/adcp/reporting/feed/store.py
Comment thread tests/conformance/reporting/test_reporting_feed_migration.py
Comment thread tests/conformance/reporting/test_reporting_feed_transactions.py

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve — clean subsequent pass, no blocking findings.

This is a subsequent review atop a prior approve. The reviewer examined the corrective child commit (a9bc445, 14 files, +174/-89) integrating main; the 38 feature paths are unchanged from the prior-approved feature. Changes are semantically neutral: -O assert-form hardening across 11 test files, a fail-closed native-feed CI selector with least-privilege permissions and a real-collation switch, a strengthened exact catalog-manifest assertion, and rc.3→rc.6/rolling-pin bumps.

Findings: none (no critical/high/medium).

Decision path: Rows 1–8 checked top-down. No critical/high findings (row 1 n/a). gated_paths is false (row 2 n/a). No (deleted) high-risk reasons — all four src/adcp/server/** matches are (modified) (row 3 n/a). No medium findings, so rows 4, 5, and 8 don't fire. Prior decision was approve, not escalate (row 6 n/a). No no-auto-approve team match (row 7 n/a). Falls through to row 9 → approve.

The high_risk flag is set only by (modified) server-path matches with no medium-or-higher concerns; per the arbiter rules this is presumed safe and is not an escalation trigger on its own. review_decision: REVIEW_REQUIRED does not block because gated_paths is false — the deprecated protected-branch gate no longer forces comment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant