Reality probe: CounterProof claim evidence can already enter PR-Agent via [artifacts]

[hippoley]

This is a Reality Probe around an existing PR-Agent feature request, not a proposal to add another review bot.

Related issue:
The-PR-Agent/pr-agent#2640

@misery @IsmaelMartinez — #2640 asks for CI-failure analysis / relatedness across many jobs. I tested one narrower composition question before proposing any integration:

can deterministic claim/evidence output be produced before the LLM step and then enter PR-Agent through the artifact path it already has?

Real handoff test

CounterProof generated a claim/evidence matrix containing scoped states such as:

• WITNESSED (submitted judge)
• UNPROVEN
• oracle alignment UNVERIFIED

I then checked out PR-Agent current main at:

eb15fc225ee39c341a1647bfb8db2139a15fbb2f

and handed the matrix in through the existing:

ARTIFACT_PATH=CLAIM_EVIDENCE_MATRIX.md

No PR-Agent patch, provider token, or model call was used.

The real inject_artifact_context() / load_artifact() path injected the CounterProof evidence into all three targets:

pr_reviewer          true
pr_description       true
pr_code_suggestions true

and the probe verified each one retained both the witnessed scope and explicit unproven rows.

Runner:
https://github.com/hippoley/CounterProof/actions/runs/35984022875

Readable receipt:
https://github.com/hippoley/CounterProof/blob/probe/pr-agent-artifact-handoff/reality/pr-agent-2640/HANDOFF.md

Why I think this might matter for #2640

Your issue already converged on an important caution: a confidently wrong “CI failure is unrelated” judgement is worse than exposing the signals the reviewer should judge from.

A possible split is therefore:

CI / replay evidence
      ↓
deterministic evidence boundary
      ↓
PR-Agent [artifacts]
      ↓
LLM explanation / review
      ↓
human reviewer

CounterProof's part would not say “this failure is unrelated” from prose alone. It would carry narrower states like:

• behavioral regression witnessed;
• infrastructure failure => inconclusive;
• claim unproven;
• oracle contradicted/unverified;
• exact BASE/HEAD/test provenance.

Question

For the 65-job GitLab case in #2640, would an artifact shaped like that be useful input to PR-Agent's review, or is the real missing value still cross-job signature dedup + provider-native job metadata, making a pre-classified claim matrix mostly beside the point?

Either answer is useful. I am specifically trying to learn whether CounterProof should compose with existing reviewers or stay out of this layer.