Reality probe: can CounterProof act as a verifier input to Codex PROVE?

[hippoley]

Neighboring project:
https://github.com/yehyakin/codex-prove

@yehyakin — I read the v1 orchestration contract because CounterProof is converging on a narrower evidence problem and I wanted to check whether we are duplicating your work.

My current read is that the projects are adjacent rather than competing:

• Codex PROVE owns REQ-IDs, routing, ownership, final-candidate binding, “verify the verifier,” and the Controller's PASS / FIX / BLOCKED decision.
• CounterProof asks a narrower question: for one claimed property, what does the executable evidence actually establish on BASE vs HEAD, did the judge change, and is the product oracle aligned / contradicted / unverified?

I wrote one manual handoff before proposing any code:

https://github.com/hippoley/CounterProof/blob/reality/codex-prove-handoff/reality/codex-prove/HANDOFF.md

Example

If the Controller predeclares:

REQ-1 requires a red→green regression
REQ-2 requires a write-failure injection test

CounterProof could return:

REQ-1
  submitted-test evidence  WITNESSED
  oracle alignment         UNVERIFIED
  overall                  WITNESSED (submitted judge)

REQ-2
  submitted-test evidence  UNPROVEN
  oracle alignment         UNVERIFIED
  overall                  UNPROVEN

The Controller would still own the judgment:

• REQ-1 may satisfy its predeclared evidence requirement if red→green is all that was required;
• REQ-2 remains unsatisfied;
• if REQ-1 required an authoritative product oracle, WITNESSED (submitted judge) would also be insufficient.

So the proposed boundary is:

CounterProof
  classifies evidence scope
       ↓
PROVE Controller
  compares it with predeclared REQ-ID evidence requirements
       ↓
PASS / FIX / BLOCKED

CounterProof would never approve the overall task.

Question

Does that fit the “verify the verifier” contract you intended in Codex PROVE?

More specifically, would you model this external verifier output as:

1. ordinary Evidence: attached to a Requirement ID; or
2. a separate evidence-quality packet the Controller must validate before accepting the underlying test/build result?

If the answer is “neither — PROVE already covers this cleanly,” that is useful too. I would rather compose with an existing evidence model than invent a second orchestration layer.

[yehyakin]

Thanks for looking into PROVE. I think the boundary you described makes sense.

I’d lean toward option 1: attach the report to the relevant REQ-ID as Evidence, with the commit IDs, test results and scope limitations included. The Controller would still check whether that evidence satisfies the original requirement and make the final decision.

I’m simplifying PROVE at the moment, so I’d keep this optional. Missing evidence should usually mean running a targeted check, not adding another approval step.

If you end up with a runnable example of this handoff, feel free to share it here. I don’t think we need a separate packet type or integration layer yet.

[hippoley]

@yehyakin — I followed your suggestion literally and kept the handoff deliberately boring.

Runnable example:
#53

It does only this:

1. generate an existing CounterProof claim/evidence artifact;
2. attach that artifact to PROVE's existing REQ-ID Requirement coverage / Verification / Evidence result shape;
3. carry BASE/HEAD, exact tests, digest, and scope limitations;
4. leave satisfied / unsatisfied / blocked and final PASS / FIX / BLOCKED entirely with the PROVE Controller.

No new packet type, adapter protocol, or approval step.

The example deliberately shows one requirement that may have enough red→green evidence and another that stays UNPROVEN, so the handoff cannot collapse into “CounterProof says PASS.”

If that is the composition you had in mind, I’ll treat this as an example/integration boundary rather than growing a dedicated PROVE feature.

[hippoley]

@yehyakin — I have one narrower follow-up before adding any integration code.

Your current PROVE contract already says evidence must bind to the final candidate and becomes stale when the candidate changes. CounterProof receipts already carry an exact head_sha / digest.

So the next useful test seems to be a stale-evidence case rather than a new adapter:

REQ-1 requires a red→green regression

CounterProof receipt:
  head_sha = abc123
  submitted evidence = WITNESSED

PROVE final candidate:
  def456

My expectation is that the old receipt must not satisfy REQ-1 until the evidence is rerun/rebound to def456, even if the changed files look unrelated.

The question is: does PROVE's existing candidate-binding contract already make that sufficient, or is there a missing machine-checkable field/shape in the external Evidence entry?

If the existing contract already covers it cleanly, I would rather document one acceptance example and build nothing. If it does not, that gap is a much better integration target than a CounterProof-specific packet.

I also merged #55 so the runnable handoff now explicitly separates Worker task status: PASS from the Controller's final PASS / FIX / BLOCKED decision.