Skip to content

fix(reasoning): detect bare READY markers and honor the last decision - #6754

Open
Kyou12138 wants to merge 2 commits into
crewAIInc:mainfrom
Kyou12138:fix/reasoning-ready-marker-detection-6204
Open

fix(reasoning): detect bare READY markers and honor the last decision#6754
Kyou12138 wants to merge 2 commits into
crewAIInc:mainfrom
Kyou12138:fix/reasoning-ready-marker-detection-6204

Conversation

@Kyou12138

Copy link
Copy Markdown

Summary

Fixes #6204.

Reasoning plan refinement only treated the full instructional phrase READY: I am ready to execute the task. as ready. Local/Ollama models often conclude with a short line such as READY / READY. / NOT READY, so the executor never left the refine loop even when the model had clearly finished planning.

Approach

Introduce response_indicates_ready() with:

  1. Line-anchored markersREADY / NOT READY (optional trailing punctuation), case-insensitive
  2. Last marker wins — after multi-step refinement, the final decision is used (NOT READYREADY, or the reverse)
  3. Full instructional phrase as fallback — only when no line marker is present
  4. No mid-sentence false positivesI am ready to begin researching… is not treated as ready

This is intentionally stricter than “substring contains READY”, which would mis-handle NOT READY.

Tests

lib/crewai/tests/utilities/test_response_indicates_ready.py covers:

  • full instructional phrase
  • bare READY / READY. / READY: …
  • NOT READY must not match as ready
  • last-marker-wins (both directions)
  • mid-sentence “ready” ignored
  • empty / unmarked plans

How to test

cd lib/crewai
pytest tests/utilities/test_response_indicates_ready.py -q

Notes / related PRs

Several community PRs also target #6204. Differentiation of this PR:

  • explicit last-marker-wins semantics for refine loops
  • NOT READY overrides a preceding full instructional READY phrase
  • dedicated unit module with the cases above

Happy to close in favor of another PR if maintainers prefer a different approach.

Planning readiness only matched the full instructional phrase
"READY: I am ready to execute the task.", so models that conclude with
a short-form READY line (common with Ollama/local models) were stuck in
endless refine loops.

Introduce response_indicates_ready() that accepts the full phrase and
line-anchored READY / NOT READY markers, with the last marker winning.

Closes crewAIInc#6204
@coderabbitai

coderabbitai Bot commented Jul 31, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 759a1fb9-e9f4-4c4e-957d-9d8cda4bbfd7

📥 Commits

Reviewing files that changed from the base of the PR and between ebe0082 and 324ac75.

📒 Files selected for processing (2)
  • lib/crewai/src/crewai/utilities/reasoning_handler.py
  • lib/crewai/tests/utilities/test_response_indicates_ready.py

📝 Walkthrough

Walkthrough

The reasoning handler adds shared detection for READY and NOT READY markers. Three response paths use this detector. Tests cover marker formats, precedence, false positives, empty responses, and planning parser integration.

Changes

Readiness detection

Layer / File(s) Summary
Detector and response integration
lib/crewai/src/crewai/utilities/reasoning_handler.py
Adds case-insensitive marker detection with last-marker precedence and updates function-calling, fallback, and static response parsing paths.
Readiness behavior validation
lib/crewai/tests/utilities/test_response_indicates_ready.py
Tests full phrases, line markers, punctuation, case variations, marker precedence, empty responses, and planning parser readiness results.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: detecting bare READY markers and honoring the final readiness decision.
Description check ✅ Passed The description directly explains the readiness detection bug, implementation approach, tests, and linked issue.
Linked Issues check ✅ Passed The changes satisfy issue #6204 by detecting standalone READY output and correctly handling readiness markers during plan refinement.
Out of Scope Changes check ✅ Passed The code and tests remain focused on readiness detection and planning response parsing described in issue #6204.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Reasoning plan always detects "NOT READY" even though the model indicates "READY"

1 participant