fix(harness): stop masking lead lane failures as population binding mismatches - #187
Merged
Merged
Conversation
…ismatches When the lead analyzer failed for its own reason, populationDiagnostics reconciled lead-side counts from absent data, reported a conflict, and collectEvidenceBundle replaced the lane with a generic SESSION_POPULATION_BINDING_MISMATCH envelope. The bundle still failed closed, but the real failure code never reached the reader, which left reports like QoderAI#164 undiagnosable. An unobserved lead lane now keeps its own failure on the lane. Binding reconciliation skips lead-attributed checks through a new optional leadObserved flag on validateSessionPopulationBundle (default true, so existing callers keep their behavior), the diagnostics record leadObserved and expose the concrete reconciliation errors array on genuine conflicts, and the fail-closed SESSION_POPULATION_BINDING_MISMATCH downgrade still applies whenever the conflict was observed against lead data. Validated with a new regression test that was verified red on the pre-fix implementation, the extended conflict assertions, and the full root suite (1771 tests). Spec: docs/specs/2026-09-23-164-lead-failure-passthrough.md, story QoderAI#164. Co-authored-by: ZCode (GLM 5.3) <zcode@z.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
An evidence bundle whose lead analyzer fails for its own reason now keeps the
lead lane's original error code. Binding reconciliation no longer manufactures
lead-side errors from absent data, genuine conflicts expose their concrete
reconciliation errors, and the fail-closed contract is unchanged.
Why
Issue/Story: [Bug]: lead lane always fails with SESSION_POPULATION_BINDING_MISMATCH on Claude while sessionEvidence succeeds #164
User or maintainer outcome: on that report, the lead lane is
"unconditionally downgraded with
SESSION_POPULATION_BINDING_MISMATCH"while
harness analyzesucceeds on the same workspace. Investigation showsthe bundle relabels any lead lane failure as a binding mismatch:
populationDiagnosticsreconciles lead-side counts from missinglead.data(every count compares as
-1), reportsconflict, andcollectEvidenceBundlethen overwrites the lane — the original error codeand message are discarded before they can reach the reader. The bundle stays
fail-closed, but the visible code points at the binding contract instead of
the real cause, which makes [Bug]: lead lane always fails with SESSION_POPULATION_BINDING_MISMATCH on Claude while sessionEvidence succeeds #164 undiagnosable from the outside.
This change makes the real failure observable again. The remaining
provider-specific cause behind [Bug]: lead lane always fails with SESSION_POPULATION_BINDING_MISMATCH on Claude while sessionEvidence succeeds #164 (likely Windows path handling; on Linux
synthetic homes the frozen population, Session lane, and lead always
reconcile) is not fixed here — it stays fail-closed but diagnosable once
this lands.
Traceability and Scope
docs/specs/2026-09-23-164-lead-failure-passthrough.mdboundwithleadObserved: false; bundle status staysfailed.the fail-closed
SESSION_POPULATION_BINDING_MISMATCHdowngrade, and thediagnostics add the concrete reconciliation
errorsplusleadObserved: true.validateSessionPopulationBundlegains an optionalleadObservedflag (default
true); existing callers keep their behavior.scripts/session-analysis/session-population.mjs,scripts/harness-analysis/evidence-bundle/index.mjs, focused tests intest/reporting/better-harness-evidence-bundle.test.mjs, new spec underdocs/specs/.lane envelope shapes, discovery or qualification behavior, report schemas,
or the underlying Windows-specific lead failure from [Bug]: lead lane always fails with SESSION_POPULATION_BINDING_MISMATCH on Claude while sessionEvidence succeeds #164.
Change Type
Test and Review Evidence
npx vitest run test/reporting/better-harness-evidence-bundle.test.mjs test/sessions/session-population.test.mjsnpm test(root suite)npx vitest run test/skills-docs/doc-link-graph.test.mjs(new spec on disk)SIMULATED_PROVIDER_CRASHfailed, diagnosticsbound+leadObserved: false,lead.error.code = SIMULATED_PROVIDER_CRASHbound,leadObserved: true, lead availableManual or visual evidence: none (no rendered surface changed).
Risk and Recovery
(
leadObserved,errorson conflict); lane envelope shapes unchanged. Theonly behavioral delta is the previously-masked path, which now surfaces the
original error code instead of the generic one. No OS-specific code touched;
verified on Linux, exercised through platform-neutral unit contracts.
list, schema version, or generated artifact changes).
return to the masked code.
masked code for unobserved lead lanes would now see the original code; no
such consumer exists in this repository (the code is only produced, never
parsed). The Windows-specific lead failure in [Bug]: lead lane always fails with SESSION_POPULATION_BINDING_MISMATCH on Claude while sessionEvidence succeeds #164 remains open.
AI Involvement
tests were produced by ZCode (GLM 5.3) under the fork owner's direction; the
fork owner approved the change and this PR text. Red-green verification and
the suite runs above are the recorded evidence.
Checklist
AGENTS.md,CONTRIBUTING.md, and the relevant canonical-owner guidance.CHANGELOG.md(not edited, following the repository's change-scope rule; happy to add an entry if maintainers prefer).