fix(core): make web search limits non-fatal and report real errors - #50938
Open
rekram1-node wants to merge 2 commits into
Open
rekram1-node wants to merge 2 commits into
rekram1-node wants to merge 2 commits into
Conversation
Oversized provider responses failed the whole search: WebSearchMcp.call rejected bodies over 256 KB, and a single 424 KB Firecrawl result was enough. The tool then replaced the cause with a generic message. All providers now read at most 1 MiB, parse a cut-off body as partial JSON, and keep the results that arrived complete. WebSearch.query caps each result's content at 4,000 characters so one page cannot crowd out the rest of the bounded tool output. Web search errors now carry readable messages, including the provider's own explanation for non-2xx responses, MCP isError results, and JSON-RPC errors, which previously surfaced as no results.
OpenCode Web Search, the default search provider for Console-connected
users, made its own request with no size cap and discarded the body of
failed responses. It now uses the shared response handling, and OpenCode
API error bodies ({ _tag, message }) supply the failure explanation.
Reading a failed response for its explanation is bounded to one second
so a stalled error body still fails promptly.
Also formats the earlier changes with Prettier and tidies the tests.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Type of change
What does this PR do?
A
websearchcall failed with onlyUnable to search the web for <query>. Firecrawl had returned 450 KB, 424 KB of it one Hacker News thread, andWebSearchMcp.callfails any response over 256 KB. The tool then replaced that error, like every error without an HTTP status, with the generic message.Limits no longer fail a search. All six providers (Exa, Firecrawl, Parallel, Tavily, TinyFish, and OpenCode Web Search) now read through
plugin/websearch/response.ts:The per-result cap keeps one huge page from pushing the other results out of the 50 KiB tool output; 10 results at the cap still fit. Measured across providers, only Firecrawl (up to 424 KB) and some Exa results (up to 8 KB) exceed it. Parallel sends
structuredContentafter a text copy of the same JSON, so a cut response falls back to the text copy.Errors say what happened. The web search errors now have messages, and the tool reports
Unable to search the web for <query> (<provider>): <reason>. MCPisErrorresults and JSON-RPC errors now fail instead of becoming "No search results found". Results from real providers:Unable to search the web for …No search results found…… (firecrawl): You've hit Firecrawl's free MCP rate limit… Fix: Create an API key at https://www.firecrawl.dev/app/api-keys…No search results found…… (exa): web_search_exa error (401): Invalid API keyWeb search authentication failed (HTTP 401)… (tavily): HTTP 401: Unauthorized: missing or invalid API key.Web search authentication failed (HTTP 401)… (tinyfish): HTTP 401: Free daily Search quota used (50/50). Sign up for continued access: https://agent.tinyfish.ai/sign-up…Unable to search the web for …… (exa): web_search_exa request timed outMessages never use the HTTP client's own error text, because it includes the request URL and Exa passes its key in the query string. The 429 failover still reads the status and
Retry-Afterfrom the sameHttpClientError.Web search tool failures no longer attach the underlying error.
toSessionErrorshows an attached error's message in place of the tool's, which is why the generic text only appeared when that message was empty. Permission errors are still attached and unwrap as before.How did you verify your code works?
Real sessions: the dev TUI with a private server from this branch (
bun dev --standalone) and Claude Sonnet 5 callingwebsearch, pinned to each provider through project config:Service-level runs against all five external providers without keys and with invalid keys, including Firecrawl's 1.1 MB response cut at 1 MiB.
New tests:
isErrorand JSON-RPC errors503{ _tag, message }bodyThe oversized Firecrawl test fails on
v2withfirecrawl_search response exceeded 262144 bytes. The existing test that a rejected OpenCode response is closed promptly still passes.packages/core: 5461 pass. The 8 failures are the same shell-tool and test-environment tests that fail onv2.sdk's "embedded client exposes plugin-backed web search" also fails identically onv2.bun run check, Prettier, andlint:effect-patternsare clean for the changed files.Checklist