Skip to content

fix(accumulator): warn on malformed streamed tool arguments - #2844

Open
mikemikimike wants to merge 6 commits into
agentscope-ai:mainfrom
mikemikimike:codex/2841-tool-call-parse-diagnostics
Open

fix(accumulator): warn on malformed streamed tool arguments#2844
mikemikimike wants to merge 6 commits into
agentscope-ai:mainfrom
mikemikimike:codex/2841-tool-call-parse-diagnostics

Conversation

@mikemikimike

Copy link
Copy Markdown
Contributor

Summary

  • warn when accumulated streamed tool-call arguments cannot be parsed as JSON
  • preserve the existing {} content fallback and merged-argument behavior
  • add regression coverage for malformed arguments emitted by a model

Testing

  • mvn -pl agentscope-core -Dtest=ToolCallsAccumulatorTest test
  • mvn -pl agentscope-core test
  • mvn -pl agentscope-core spotless:check
  • git diff --check

Closes #2841

@gilgold

gilgold commented Aug 26, 2026

Copy link
Copy Markdown

The added observability is useful, but I think this exact form creates two safety problems:

  1. log.warn(... rawContentStr, e) puts the full tool payload into default telemetry. Tool arguments routinely contain credentials, customer text, file contents, and prompt-injection material. I’d log only safe metadata by default: tool-call id/name, provider/model if available, byte length, SHA-256, parser exception class/offset, and stream finish reason. Any raw sample should require an explicit debug policy and be redacted + size-capped.
  2. The new test codifies a more serious ambiguity: after the final JSON parse fails, previously merged input remains executable while content becomes {}. That can combine stale/partial fields with a malformed final payload and makes a parse failure indistinguishable from a genuinely valid empty object downstream.

A minimal fail-closed contract would be:

  • mark the block with a typed PARSE_FAILED outcome (or metadata flag) distinct from valid {} and INTERRUPTED;
  • preserve raw arguments only in an ephemeral caller-controlled diagnostic channel, never checkpoints/default logs;
  • prevent tool execution whenever final parse failed;
  • if repair is enabled: strict parse → conservative deterministic repair → tool-schema validation → execute. Unescaped quotes are ambiguous, so if schema validation cannot prove one reconstruction, return the typed error or make one bounded model retry rather than guessing;
  • change the regression to assert malformed input cannot reach the executor as retained partial args, while valid {} still can.

For AgentScope, this belongs locally in the accumulator/JsonCodec seam. Disclosure: I maintain JSONSuture, a hosted option for teams that need one repair + schema-validation boundary across Java/Python/Node services; it is not a replacement for the framework-level fix here.

@mikemikimike

Copy link
Copy Markdown
Contributor Author

Addressed in 6106aa9: malformed tool-call diagnostics now log only tool id/name, UTF-8 byte length, SHA-256 digest, exception type, and message. The raw argument payload is no longer emitted to default logs. Existing {} fallback and merged-argument behavior remain unchanged. Focused tests (16/16), module tests, Spotless, and git diff --check pass. The earlier full CI failure was unrelated, in agentscope-harness teardown.

@codecov

codecov Bot commented Aug 26, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.50000% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...e/core/agent/accumulator/ToolCallsAccumulator.java 87.50% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

@gilgold

gilgold commented Aug 26, 2026

Copy link
Copy Markdown

Thanks — 6106aa9e resolves the default-log payload exposure cleanly.

The execution-integrity issue is still present, though, and the new regression currently locks it in: input.path=output.txt remains executable even though the final accumulated payload is malformed and content is emitted as {}. Unless a later layer has an independent parse-failure guard, write_text_file can still run with stale/partial arguments while downstream code cannot distinguish that state from a valid empty object.

A minimal follow-up can stay local and avoid choosing a repair policy here:

  • set a typed PARSE_FAILED outcome/metadata flag in the catch path;
  • clear or quarantine executable input for that block;
  • have the executor reject PARSE_FAILED before tool dispatch;
  • change this regression to assert malformed final JSON is non-executable, while a genuinely valid {} remains executable.

If changing ToolUseBlock is too broad for this PR, a dedicated follow-up issue is reasonable, but I would not merge the retained-input test as the intended safety contract.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: ToolCallsAccumulator silently swallows JSON parse failures for streamed tool call arguments and the raw string is lost

2 participants