fix(llm): don't swallow response_model conversion errors on streaming (#6735)

[Anai-Guo]

Problem

Fixes #6735.

When response_model is set on a streaming call, _handle_streaming_response runs the
InternalInstructor.to_pydantic() / model_dump_json() conversion inside the try
that wraps chunk consumption. The except Exception below that try exists to salvage a
partial response from a stream that broke mid-flight — it returns full_response whenever
there is text to return.

As a result, a conversion failure is routed through the salvage path: a caller who asked
for a BaseModel-shaped result gets the raw prose back, with no exception and no signal
that conversion was attempted and failed.

A conversion failure is not a broken stream. The stream completed successfully; the output
just didn't match the schema. Routing it through the salvage path conflates the two.

Fix

Wrap the response_model conversion in its own try/except that raises a dedicated
StructuredOutputConversionError, and re-raise that error before the salvage except Exception
— mirroring the existing except LLMContextLengthExceededError: raise handling in the same
method. Genuine mid-stream breaks still fall through to the salvage path unchanged.

Scope note

The issue also references the async handler. On main, _ahandle_streaming_response does not
currently perform any response_model conversion (it returns full_response directly), so there
is no conversion-swallow path to fix there today. This PR therefore targets the confirmed sync
path; the async streaming handler ignoring response_model entirely is a separate gap and out of
scope here.

Testing

Added test_streaming_response_model_conversion_failure_raises, which drives a clean stream whose
text fails conversion (to_pydantic patched to raise) and asserts the method now raises
StructuredOutputConversionError instead of returning the prose.

Verified locally against crewai 1.15.9:

• Before: the method returned 'just some prose' (raw text, no exception).
• After: it raises StructuredOutputConversionError.
• The success path (conversion succeeds) still returns the structured JSON unchanged.
• ruff 0.15.1 check and format are clean on the changed source.

🤖 Generated with Claude Code

[coderabbitai]

📝 Walkthrough

Walkthrough

Streaming response-model conversion failures now raise StructuredOutputConversionError instead of being treated as broken streams and returning raw text. A regression test verifies this behavior for prose-only streaming chunks and a failed Pydantic conversion.

Changes

Structured streaming errors

Layer / File(s)	Summary
Conversion error routing lib/crewai/src/crewai/llm.py	Adds StructuredOutputConversionError, wraps structured conversion failures with it, and re-raises it before partial-response salvage handling.
Conversion error validation lib/crewai/tests/test_llm.py	Adds imports and a regression test confirming failed streaming response-model conversion raises the dedicated exception.

Suggested reviewers: greysonlalonde, lorenzejay

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Linked Issues check	⚠️ Warning	Issue #6735 calls for both streaming handlers, but this PR only changes the sync handler and leaves the async path unmodified.	Extend the conversion guard to _ahandle_streaming_response and add regression coverage for async streaming behavior.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly describes the main fix: not swallowing streaming response_model conversion errors.
Description check	✅ Passed	The description is directly about the same bug, fix, and test coverage in this PR.
Out of Scope Changes check	✅ Passed	The changes stay focused on streaming response_model conversion handling and the matching regression test.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lib/crewai/tests/test_llm.py`:
- Around line 1224-1228: Update the test around llm._handle_streaming_response
to capture the StructuredOutputConversionError via pytest.raises as exc_info,
then assert the wrapped exception’s message matches the preserved original
message and its __cause__ is the underlying conversion error. Keep the existing
response_model=_Answer setup and invalid response input unchanged.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 21c8409b-3c9b-478c-8977-6c4028995cd6

📥 Commits

Reviewing files that changed from the base of the PR and between ebe0082 and 0bf7a5e.

📒 Files selected for processing (2)

• lib/crewai/src/crewai/llm.py
• lib/crewai/tests/test_llm.py

[coderabbitai]

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Assert the wrapped error contract, not only its type.

The implementation preserves the original message and chains the conversion error, but this test would still pass if either behavior regressed. Capture exc_info and assert the message and __cause__.

As per coding guidelines, unit tests should focus on observable behavior.

Suggested assertion

-        with pytest.raises(StructuredOutputConversionError):
+        with pytest.raises(
+            StructuredOutputConversionError, match="schema mismatch"
+        ) as exc_info:
             llm._handle_streaming_response(
                 {"messages": [{"role": "user", "content": "hi"}]},
                 response_model=_Answer,
             )
+        assert isinstance(exc_info.value.__cause__, ValueError)

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	with pytest.raises(StructuredOutputConversionError):
	llm._handle_streaming_response(
	{"messages": [{"role": "user", "content": "hi"}]},
	response_model=_Answer,
	)
	with pytest.raises(
	StructuredOutputConversionError, match="schema mismatch"
	) as exc_info:
	llm._handle_streaming_response(
	{"messages": [{"role": "user", "content": "hi"}]},
	response_model=_Answer,
	)
	assert isinstance(exc_info.value.__cause__, ValueError)

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/crewai/tests/test_llm.py` around lines 1224 - 1228, Update the test
around llm._handle_streaming_response to capture the
StructuredOutputConversionError via pytest.raises as exc_info, then assert the
wrapped exception’s message matches the preserved original message and its
__cause__ is the underlying conversion error. Keep the existing
response_model=_Answer setup and invalid response input unchanged.

Source: Coding guidelines