feat(reporting): persist buyer receipt submission intents - #1219
Conversation
️✅ There are no secrets present in this pull request anymore.If these secrets were true positive and are still valid, we highly recommend you to revoke them. 🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request. |
003e7d3 to
c90274c
Compare
Keep the full core selection with a 30-minute job budget and preserve the process lane at 15 minutes. Address buyer submission review comments on redacted errors and test assertions.
bokelley
left a comment
There was a problem hiding this comment.
Review verdict: REQUEST_CHANGES (posted as a comment — GitHub blocks requesting changes on your own PR; reviewer session 6bcadd0c)
Reviewed c90274c2 (current PR head). Ran make lint (pass) and the buyer slice locally against PostgreSQL 16.
P2-1 — validate_submission re-validates the entire confirmed history on every read and write
models.py:347-351 calls _restore_confirmation for every already-confirmed chunk each time a submission is validated, and that path re-does full response schema validation. It compounds: _restore_confirmation (models.py:431) rebuilds a kinds map from all plan items per chunk, and _confirmation (models.py:370) re-parses the whole plan to reach one request.
Measured on this head (memory store, full submit):
| receipts | seconds |
|---|---|
| 100 | 0.54 |
| 500 | 9.28 |
| 1000 | 36.5 |
| 2000 | 149.3 |
~4x the time for 2x the items. PgReportingSubmissionIntentStore.confirm runs this inside the FOR UPDATE scope transaction, so scope lock hold grows with history length — that is the part that affects other writers, not just this caller.
P2-2 — schema readiness does not verify the seven shipped CHECK constraints
pg.py:96-116: required covers only the two primary keys and the foreign key. The DDL ships seven CHECKs (reporting_buyer_scope_digest, scope_bound, submission_id, plan_digest, confirmed_digest, plan_bound, confirmed_bound). Because the tables are created with CREATE TABLE IF NOT EXISTS, a pre-existing table that is missing or has weakened any of them passes create_schema — which is exactly the case this check exists to catch. The query also selects conname without the owning table, so an identically named CHECK on the wrong table would satisfy it.
CI — core Postgres lane exceeds its budget
ci.yml:138 sets timeout-minutes: 15 for both lanes. In run 36128723991 the core job was killed at 15m18s having reached 58% (1249 passed, 0 failed — the 38 ERROR lines in that log are PostgreSQL server-log lines from negative-path tests, not pytest errors). The buyer cells added here account for 5m45s of that 15-minute lane, 38.3% of it. There is no green full-CI run on this head; the newer green runs are PR Title Check only.
All three already appear fixed on the local branch (5e1789f1, 0a0fb51f) — that work simply is not pushed, so the PR head is still c90274c2. Push and re-run CI. I verified the slice source in 5e1789f1 is byte-identical to the correction I reviewed in detail, and the bounded byte-keyed validation caches, the snapshot/CAS split that moves validation out of the lock, and the full seven-CHECK readiness all address the two findings above correctly.
Non-blocking: encode_submission returns plan text and plan digest, but confirm discards both (pg.py:323 on the local head) — that decodes and SHA-256s up to 16 MiB per chunk for nothing.
|
Verification of the unpushed fix, for the record: ran the buyer slice (cas, migration, validation, uncertainty, seller, process, submissions) against real PostgreSQL 16 on the local branch head So the only thing standing between this PR and a clean re-review is the push. |
c90274c to
0a0fb51
Compare
There was a problem hiding this comment.
Ladon verdict: Approve
Approve.
Fully additive PR introducing the new adcp.reporting.submissions package (durable buyer receipt-submission intents) plus a CI timeout bump. No blocking or Medium findings.
Checks performed:
- No no-auto-approve team match; author gate not triggered.
- gated_paths: false, so the hard approval gate does not apply (review_decision REVIEW_REQUIRED is irrelevant when gated_paths is false).
- high_risk: false; no deleted/modified sensitive-path escalation triggers.
- Reviewer verified the two largest source files (models.py, pg.py): confirmation ID/body matching, byte-and-digest immutability round-trips, idempotent-replay normalization, and complete-snapshot CAS all fail closed; closed error codes leak no driver/SQL/wire detail.
- Semver signal compliant (feat, additive only), type-system import layering intact, no credentials in ctx_metadata, transport timeout/cancellation handling present, ci.yml gate armed.
No findings of any severity. Rows 1–8 do not fire; falls through to row 9 → approve.
Adds
adcp.reporting.submissionsso a buyer can reserve and replay the same receipt request after a timeout or process crash. A trusted authorization adapter supplies the seller, resolved account and canonical consumer identity. PostgreSQL reserves the complete mixed receipt plan before sending, limits each request to 100 items, and retains ordered successes and per-item failures. Existing checkpoint protocols remain compatible.Confirmation validates exact response IDs and bodies, preserves the confirmed prefix, and compares the complete persisted state before committing. Bounded caches hold immutable validation evidence; expensive validation runs outside the PostgreSQL scope lock. Schema readiness checks every shipped CHECK constraint as well as keys and indexes.
For rollout, supply an application-owned PostgreSQL pool and call
PgReportingSubmissionIntentStore.create_schema()before enabling submissions. After uncertainty, resume withsubmit_reporting_receipts(..., receipts=None)using the same store and authorizer. Keep the intent tables when rolling back so outstanding requests retain their identities. The memory store is a volatile reference implementation.The Postgres core CI job now has a 30-minute budget for its full conformance and installed-artifact coverage; the process lane remains at 15 minutes. Test selection and per-case deadlines are preserved. Review cleanup documents closed error handling and strengthens immutability and corruption assertions.
Validation:
make lint,make typecheck-all, andmake validate-generatedpassed.make testwith coverage: 11,715 passed, 2,182 skipped, 9 deselected, 1 existing xfail; 81.57% coverage. The standard target ran through the repository reporting harness.Refs #1167
Refs #1172