Skip to content

Reality probe: independent before/after witness for Continue #12576 #22

Description

@hippoley

This is an independent CounterProof Reality Probe for:

continuedev/continue#12576

@christianalmonte112 — you bumped that PR on 2026-09-16 because the fix is still waiting for maintainer review. I tried to answer one narrow question that may help the review: does the regression test added by the PR actually distinguish the fix from the pre-change code?

Independent replay

Exact commits:

  • HEAD: 0f2e62e54b6f5bc2bc749e22e15b0dd5342bde2a
  • BASE: a3076b444af3fb0d9fd3805c10f75177d8f70137

Changed test:

  • core/config/profile/doLoadConfig.vitest.ts

Result on a clean GitHub runner:

  • HEAD + changed test: 5/5 PASS
  • BASE + the same changed test: 4/5 PASS, 1 FAIL
  • CounterProof status: WITNESSED
  • result protocol: json-v1
  • evidence digest: sha256:23b69b350eb966d65faf9616af31eb6c961a765e4db2672083cde7ca336b6866

Runner:
https://github.com/hippoley/CounterProof/actions/runs/35972260870

So the submitted Vitest file is not merely green on the PR branch: it contains behavior that fails against the exact pre-change base.

Scope

This does not mean “the PR should merge.”

It proves the submitted migration regression distinguishes HEAD from BASE. It does not replace maintainer judgment about API/design direction, backwards compatibility, or other config shapes beyond the test coverage.

The earlier Cubic finding about arrays appears already addressed in current HEAD; this receipt is not a substitute for that review history.

Something this PR taught CounterProof

This external PR broke CounterProof twice before producing a receipt:

  1. *.vitest.ts was not recognized as a changed test.
  2. the temporary BASE worktree reused only root node_modules, while Continue resolves Vitest from core/node_modules.

Both gaps are now fixed in CounterProof main because this PR exposed them:

Question

Would an independent before/after receipt like this be useful when you are trying to get a stalled PR reviewed, or is it mostly noise compared with the project’s normal CI?

Even “useful / not useful + why” is enough — I am using real contributor/reviewer reactions to decide what CounterProof should become.

Readable receipt:
https://github.com/hippoley/CounterProof/blob/main/reality/continue-12576/WITNESS.md

Assertion-level reproduction

A second clean run reproduced the same semantic split and exposed the exact failing BASE assertion:

  • HEAD: 5/5 pass
  • BASE + same changed test: 4/5 pass, 1 fail
  • failing test: should transform a JSON-style tabAutocompleteModel object into a roles-based model entry
  • BASE failure: the passed YAML content still contains tabAutocompleteModel, exactly the behavior #12576 is intended to migrate away

Second runner:
https://github.com/hippoley/CounterProof/actions/runs/35975618130

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions