Keep Account Pooler accounts usable when upstream rejects fresh tokens - #4385
Merged
Merged
Conversation
A 401 on a freshly refreshed OAuth token (the ChatGPT outage's invalid_api_key responses) was stored as a permanent account error that nothing cleared. The hub now refreshes once on 401 or 403, and when the new token is still rejected it returns 503 and holds that token out of routing for one minute instead of marking the account. Only 400/401 refresh responses are permanent credential failures; unreadable refresh bodies and other statuses are transient. `bb pool account refresh <id>` now forces a refresh for an errored OAuth account and clears the error on success. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SawyerHood
deleted the
fix/account-pool-upstream-credential-rejection
branch
September 26, 2026 03:02
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Human comments
What was wrong
During the 2026-09-25 Codex outage,
chatgpt.com/backend-api/codexanswered every request with HTTP 401invalid_api_key("Incorrect API key provided: sk-svcac…"), a key the pooler never sends. The hub refreshed the OAuth token, the refresh succeeded, the fresh token got the same 401, andmarkAuthErrorstored a permanent account error. Stored errors are never cleared automatically: selection skips errored accounts,freshSecretrethrows the stored error before refreshing, andbb pool account refresh <id>goes through the same path. Both Codex accounts stayed errored until they were removed and logged in again. A 403 marked the account without even trying a refresh.What changed
hub.ts: 401 and 403 both trigger one forced refresh. If the upstream rejects the newly issued token, the credential is proven valid, so the hub returns 503 (with the upstream detail) and records a token-keyedupstreambackoff that holds the account out for one minute instead of marking an error. Rejection checks still guard against a late response poisoning a newer token. API-key accounts keep the old behavior.provider-adapter.ts: only HTTP 400/401 from the OAuth token endpoint are permanent refresh failures; other statuses and unreadable 200 bodies (parseOAuthRefreshResponse, used by both adapters) are transient.bb pool account refresh <id>/account.refreshUsage: for an errored OAuth account, forces a refresh past the stored error and clears it on success; a permanent failure updates the error message, and the CLI reports the account still in error.PLUGIN_OVERVIEW.mdis at the 4,000-character bundled-plugin cap).How you verified
invalid_api_keybody (503, no account error, no upstream or refresh traffic during the hold, recovery on the same token afterwards); manual refresh clears a storedinvalid_granterror; refresh 403/404 and unreadable bodies are transient. All 10 fail against the previous source and pass now.pnpm exec turbo run typecheck test --filter=bb-plugin-account-pool: 323 passed.🤖 Generated with Claude Code