Skip to content

fix(agent): skip invalid JSON candidates - #2683

Merged
webbrain-one merged 3 commits into
webbrain-one:mainfrom
alectimison-maker:fix/skip-invalid-json-candidates
Aug 6, 2026
Merged

fix(agent): skip invalid JSON candidates#2683
webbrain-one merged 3 commits into
webbrain-one:mainfrom
alectimison-maker:fix/skip-invalid-json-candidates

Conversation

@alectimison-maker

@alectimison-maker alectimison-maker commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Problem

Planner and classifier responses can contain an illustrative or malformed object before the intended JSON result. The extractor stopped after either a balanced candidate that failed JSON.parse or an opening brace that never closed, so a later valid result was discarded.

Change

  • continue scanning after a balanced candidate fails JSON.parse
  • resynchronize after an unterminated opening brace, capped at 16 restarts
  • preserve fenced-output priority, string escaping, and first-valid-object behavior
  • mirror the implementation across Chrome and Firefox
  • add malformed-candidate, unbalanced-prefix, bounded-work, and planner regressions

Scope and consumers

The extractor serves the planner, progress-intent, safety-refusal, plan-only, and visible-media location paths. Each consumer still applies its existing shape checks; this change only makes a later parseable JSON value reachable. An earlier valid but semantically unrelated object still wins and can cause the consumer to return null; choosing the first value that normalizes is intentionally out of scope.

Validation

  • node test/run.js ? 1,453 passed; the only failure is the pre-existing package.json 26.0.10 vs CHANGELOG.md 26.0.0 assertion
  • node test/security/injection-corpus.mjs ? 60/60 passed
  • Chrome and Firefox extractor modules are byte-identical
  • node --check for both extractor modules ? passed
  • git diff --check ? passed

@vercel

vercel Bot commented Aug 4, 2026

Copy link
Copy Markdown

@alectimison-maker is attempting to deploy a commit to the esokullu's projects Team on Vercel.

A member of the Team first needs to authorize it.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates shared planner/classifier JSON extraction to skip malformed balanced objects and continue to the first valid object.

Changes:

  • Continues scanning after JSON.parse failures.
  • Mirrors behavior across Chrome and Firefox.
  • Adds regression coverage for both planners.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
src/chrome/src/agent/json-extract.js Adds resilient candidate scanning.
src/firefox/src/agent/json-extract.js Mirrors Chrome parser behavior.
test/run.js Tests malformed-candidate recovery.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Skipping unparseable candidates covered the illustrative-object case but
not the neighbouring one: a `{` that never closes still abandoned the
whole candidate, so a stray brace in prose ahead of the real object
discarded it. Scanning now resumes one character past an unbalanced
opener, capped at 16 restarts so a "{{{{..." response stays linear.

This mirrors the scanner in tool-call-parser.js so the two agree on
recovery, and adds coverage for the brace flood plus the still-standing
limitation that an earlier *valid* non-plan object wins.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@esokullu

esokullu commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

Pushed a commit extending the recovery to one more case. No Copilot findings on this one — its review was a summary only.

What I added. Skipping unparseable candidates covered the illustrative-object case but not its neighbour: a { that never closed still abandoned the whole candidate, so a stray brace in prose ahead of the real object discarded it. Scanning now resumes at start + 1 on an unbalanced opener, capped at 16 restarts to keep a pathological "{{{{…" response linear.

This is the same fix as #2682, deliberately. The two scanners had drifted apart on exactly this behaviour — each handled one half of the problem — which is worth knowing about since it's how the same bug ended up in both trees. They agree now.

Worth noting for reviewers, since the description mentions only the planner: this function also backs three classifier paths — agent.js:11454 (progress-intent), _isSafetyRefusalTerminal, and _looksLikePlanOnlyTerminal. I walked all three and none looks like a downgrade; every consumer shape-checks (normalizePlan requires summary, the refusal detector requires confidence === 0 plus refusal wording, the plan-only check requires ≥5 policy keys), and for the refusal detector the change makes a refusal more likely to be caught, which is the safe direction. Still, the blast radius is wider than "planner" suggests.

Tests. An unbalanced opener followed by two broken candidates then the real plan; the bounded brace flood; and one pinning the limitation this PR correctly scopes out — an earlier valid non-plan object still wins and returns null. That last one asserts current behaviour on purpose, so if anyone later moves to "first object that normalises," the test fails and forces the decision to be explicit rather than silent.

1453 passed, security corpus 60/60, Chrome and Firefox copies cmp-verified identical. The one failure is the pre-existing package.json 26.0.10 vs CHANGELOG 26.0.0 mismatch.

@alectimison-maker

Copy link
Copy Markdown
Contributor Author

Thanks. I reviewed the current head (11c1431e) and agree that recovery from an unterminated opener belongs with the invalid-candidate fix. Restarting at the next candidate, with the 16-restart cap, preserves recovery without leaving malformed brace floods unbounded.

I also traced the extractor's planner, classifier/terminal, and visible-media consumers. Their existing shape checks remain in place, so the scanner change makes later parseable JSON reachable but does not make an arbitrary object a valid plan, refusal, plan-only result, or media location. The known first-valid-object limitation remains unchanged.

I've updated the PR description to include the unbalanced-opener behavior and the wider consumer set. Local validation is 1,453 passing runner tests with only the existing package/CHANGELOG mismatch, 60/60 security-corpus checks, syntax checks, Chrome/Firefox parity, and git diff --check.

@webbrain-one
webbrain-one merged commit c052b9f into webbrain-one:main Aug 6, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants