Skip to content

fix(langchain): capture structured output tool calls - #1428

Open
yaodong-shen wants to merge 1 commit into
AgentOps-AI:mainfrom
yaodong-shen:fix/1263-langchain-structured-output
Open

fix(langchain): capture structured output tool calls#1428
yaodong-shen wants to merge 1 commit into
AgentOps-AI:mainfrom
yaodong-shen:fix/1263-langchain-structured-output

Conversation

@yaodong-shen

Copy link
Copy Markdown

📥 Pull Request

📘 Description

Fixes #1263.

LangChain's with_structured_output() returns an AIMessage whose text is often empty while the structured result is stored in tool_calls. The callback previously serialized only generation text, so AgentOps traces could not render structured outputs correctly.

This change:

  • emits completions through AgentOps' indexed gen_ai.completion.{i} semantic attributes;
  • captures normalized LangChain AIMessage.tool_calls and the legacy OpenAI-shaped additional_kwargs.tool_calls form;
  • preserves text completions, roles, finish reasons, tool metadata, and JSON arguments without double-encoding;
  • adds regression coverage for text, normalized structured output, and legacy tool-call responses.

🧪 Testing

  • pytest tests/unit -q -o log_cli=false — 491 passed, 1 skipped
  • focused LangChain utility tests — 3 passed
  • Ruff 0.12.9 check and format — passed
  • pre-commit on all changed files — passed
  • real langchain_core AIMessage callback smoke test — passed
  • git diff --check — passed

@gnanirahulnutakki gnanirahulnutakki left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed and reproduced on exact head 85f689612f3fff70274af24739bb22b522455418.

I verified the structured-output path with a real langchain_core AIMessage/ChatGeneration: normalized tool-call arguments are serialized once and emitted under the existing indexed gen_ai.completion.0.tool_calls.0.* conventions, while text completions retain role and finish-reason attributes.

Local verification:

  • pytest tests/unit -q -o log_cli=false: 491 passed, 1 skipped
  • focused LangChain utility tests: 3 passed
  • Ruff 0.12.9 check and format check on all changed files: passed
  • git diff --check: passed

No blocking issues found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Broken viz in Agent Ops Traces when using with_structured_output

2 participants