🤖 feat: generalize Coder AI Gateway integration to arbitrary provider instances - #3844
Conversation
…nstances The Coder integration previously hardcoded two provider names (anthropic, openai) for both model discovery and request routing, while AI Gateway deployments configure N provider instances with arbitrary names and types (openai, anthropic, azure, google, openai-compat, openrouter, vercel, bedrock, copilot), each mounted at /api/v2/aibridge/<name>/. Any other provider was undiscoverable and unroutable, and discovery only ever ran once at login. - Discovery now lists the deployment's configured providers via GET /api/v2/ai/providers (authoritative: name, type, enabled) and, when that admin-only endpoint is 403/404, falls back to probing the default type-named routes plus any previously known instances. - Catalog writes are per-provider conclusive: ok replaces, 404 clears, transient errors carry the provider's previous entries forward - one provider whose upstream rejects /models (e.g. Bedrock) can no longer poison every other provider's refresh. - Routing resolves coder:<name>/<model> through persisted provider metadata (discoveredProviders, then the user-managed additionalProviders escape hatch for custom names members cannot list, then the name === type default) and picks the wire protocol by type: anthropic/bedrock speak Anthropic messages, openai speaks the Responses API, other OpenAI-wire types speak chat completions. Copilot instances are rejected as unsupported (they need client-minted request-time tokens). - New coderOauth.refreshModels IPC, a Settings 'Refresh models' button, and a command palette action re-run discovery without a re-login.
…y catalogs Found while dogfooding against dev.coder.com: /v1/models is a gateway passthrough, and Anthropic rejects requests without the required anthropic-version header with a conclusive 400 — which discovery read as 'no models', hiding every Anthropic model. Send the header for anthropic- and bedrock-type provider instances only.
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4d46a948cc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…iently Codex P2: on a fresh login (discoveredModels absent) the per-provider carry-forward has nothing to carry for a transiently-failed provider, so persisting the other providers' lists would flip routing from fail-open to an authoritative partial catalog that blocks the failed provider's models. Skip the write (and surface an error from refreshModels) when the prior catalog is unknown and any provider errored; known catalogs keep the per-provider merge.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3de90766d9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…g stays unknown Codex P2 round 2: the fresh-unknown guard also discarded the authoritative /api/v2/ai/providers listing, leaving custom-named instances unresolvable (manual models unroutable) until every catalog request succeeded. The listing is conclusive independently of the catalog fetches: persist discoveredProviders while keeping discoveredModels absent (fail-open). Probe-derived fallback metadata is just the name===type default routing already applies, so the inconclusive probe path still skips all persistence.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 549fa6981f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Two findings on the same root cause — carry-forward requires prior state for the errored provider, not just a present catalog: 1. All-fetches-failed guard ran before the locked write, discarding a conclusive authoritative listing. The early return is gone; the write predicate now handles every case, persisting discoveredProviders when the listing is authoritative even if no catalog was fetched. 2. A KNOWN catalog + an admin-added provider whose first fetch errors persisted a partial catalog that blocked the new provider's models. An errored provider without prior state (no prior metadata and no prior entries under its prefix) now flips the whole catalog to UNKNOWN (fail-open), preserving manual entries and the authoritative listing; the next successful refresh rebuilds the catalog.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 22ba4d11a7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…icalize options (Codex round 4) - fromCoderGatewayModelId only canonicalizes default-named anthropic/openai instances; coder:google/x etc. stay gateway-scoped so routing never bypasses the gateway - resolveCoderWireCanonicalModel derives the wire origin from the instance type; buildProviderOptions/buildRequestHeaders use it so custom-named instances emit identical thinking/cache options and 1M beta headers - expose discoveredProviders/additionalProviders through provider IPC
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6b6874d531
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
An instance can use a canonical route name with a different type (e.g.
{name: "openai", type: "anthropic"}); normalizeToCanonical rewrote
coder:openai/<model> to openai:<model> from the name alone, so option and
header building never reached the metadata-aware wire resolution and
emitted OpenAI options for an Anthropic-wire request. Inspect the raw
Coder identity first (mirroring resolveAndCreateModel's raw-prefix shadow
check) so metadata wins over the name convention.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f53453f405
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…ry-forward (Codex round 6) - resolveAndCreateModel returns wireProviderName resolved from the RAW coder:<instance>/<model> prefix (metadata wins over name convention); aiService keys prepareProviderRequestMessages, prepareMessagesForProvider, and the providerOptions namespace key on it so Anthropic-only reasoning transforms, PDF-filename sanitization, and extras merging run against the wire the request actually speaks. canonicalProviderName remains the config identity for providers.jsonc modelParameters lookups and telemetry. - Coder catalog carry-forward on transient /models failures now requires the authoritative listing's type to match the prior provider type; a type change flips the catalog to inconclusive instead of leaving the old type's model IDs selectable on the new wire.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 91c6188fd6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 91c6188fd6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…ex round 7) Gateway-scoped coder:<instance>/<model> strings carried no catalog identity: resolveModelForMetadata/getModelStatsResolved missed known models, so budgeted goals rejected them as unpriced, context limits were unknown (no limit-driven compaction), and custom Anthropic instances failed supportsAnthropicCache. - resolveCoderMetadataCanonicalModel maps an instance's type to the canonical catalog model (anthropic/openai/google/azure/bedrock/ openrouter:<model>, vercel vendor/model; openai-compat/copilot stay gateway-scoped). Routing identity unaffected. - resolveModelForMetadata applies it after explicit mappedToModel overrides and never for a custom provider shadowing the coder prefix. - supportsAnthropicCache accepts providersConfig and resolves Coder wires from instance metadata first (raw identity, before name-based normalization); cache-control entry points thread the config through streamManager and messagePipeline.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5124d2d198
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…fig merge (Codex round 8) - The refusal-fallback rebuild now passes the RAW nextModelString to prepareMessagesForProvider, buildProviderOptions, buildRequestHeaders, and the mid-turn thinking rebuild closure (matching the main path's raw modelString). Canonicalization rewrites cross-typed Coder instances (coder:openai/x, type anthropic) to a direct-provider string, hiding instance metadata from cache/option/header builders. - _createModelCore classifies Coder models as Anthropic-routed by their resolved WIRE type (instance metadata, shadow-guarded), not the route name: custom-named Anthropic instances now honor the backend's authoritative disableBetaFeatures/cacheTtl, and cross-typed anthropic-named instances no longer receive Anthropic-only config.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 928a80e678
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
… metadataModel Round 47 Codex review fixes: - The refusal-fallback prepare path now computes only a PRELIMINARY thinking level pre-factory (xAI variant swap input, never Coder-metadata-dependent) and recomputes the FINAL nextMinThinkingLevel/nextThinkingLevel from the pinned nextProvidersConfig, mirroring the main path's split, so message and option builders agree with the wire the fallback SDK model was created for. - CompactionHandler.performCompaction carries the compaction stream's request-pinned metadataModel into the persisted summary metadata, so session-usage rebuilds after a catalog retag key/reprice the compaction request from the pinned identity instead of mutable current metadata.
|
@codex review Please take another look. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 363813b804
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…compaction defaults Round 48 Codex review fixes: - The advisor runtime's createModel now returns the wire-resolved optionsModelString derived from the SAME snapshot that created the model (mirroring resolveOptionsCanonicalModel's shadow + wire rules), and the advisor tool builds providerOptions inside execute from that identity. coder:prod-anthropic/... now gets Anthropic reasoning options, and a cross-typed coder:openai/... (type anthropic) no longer gets OpenAI-namespaced options. - AgentSession.getPreferredCompactionSettings normalizes the configured compaction default with gateway-preserving normalizeSelectedModel, so a cross-typed Coder compaction model no longer persists as the direct name-alike provider.
|
@codex review Please take another look. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e950c37e9a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…or keys Round 49 Codex review fixes: - Added AIService.createModelWithPinnedMetadata (one providers.jsonc snapshot for SDK model creation AND the pricing identity). Memory consolidation/harvest and workspace-status generation use it and pass the creation-time metadataModel into recordHeadlessUsage, so a Coder catalog refresh mid-run can no longer re-attribute their sidecar/ledger rows. - Per-model thinking-floor keys are now gateway-preserving (normalizeSelectedModel) in useMinThinkingLevels, config load normalization, agentSession's send-path override lookup, and the aiService fallback path, so an explicit coder:<instance>/<model> floor stays distinct from the direct provider's same-ID entry.
|
@codex review Please take another look. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c50a0fcec3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…ss metadata - lookupMinThinkingLevelOverride (policy.ts): gateway-preserving key first, legacy name-canonical fallback so floors persisted by older versions keep applying; used in useMinThinkingLevels, agentSession, aiService fallback path - createModelWithPinnedMetadata + advisor createModel: derive the pinned metadata identity (and advisor options identity) from the EFFECTIVE route via new ProviderModelFactory.resolveEffectiveModelString, so a coder selection that falls away to a direct provider is priced/bucketed under the route that served it
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8ed2774bff
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
hideModel/unhideModel now persist the gateway-preserving identity (normalizeSelectedModel) so hidden entries match the raw list strings that filterHiddenModels and ModelsSection compare against; name-only canonicalization left coder:<instance>/<model> visible while hiding the distinct direct model. Red-checked hook test added.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dcfd4b6f87
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
useContextSwitchWarning.handleModelChange compared selections via normalizeToCanonical, so coder:openai/<model> equaled openai:<model> and the picker returned before persisting switches between the Coder entry and the distinct direct model. Compare with normalizeSelectedModel instead. Red-checked hook test added.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f4fd662924
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
- modelSelectionEqualityKey (models.ts): keeps raw coder:<instance>/<model> identities distinct (instance TYPE decides the upstream) while still collapsing passthrough gateway aliases (mux-gateway:openai/x) to canonical. Used by modelChange.ts explicit-change tracking and checkContextSwitch same-model suppression so coder<->direct switches record user origin and get context warnings. Red-checked tests in both suites. - ModelRow accepts providersConfig and resolves stats (getModelStatsResolved) + thinking policy/floor helpers through it so custom-named Coder instance rows show pricing/context and the min-thinking selector; ModelsSection threads its config into both row tables.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ade8fa8535
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…ggestion - ProvidersSection: Refresh models click no longer void-discards the async mutation; unexpected rejections settle into coderModelRefreshState (async JSX handlers are rejected by no-misused-promises, so use explicit .catch). - suggestion.ts: getExplicitCompactionSuggestion and the authoritative catalog predicate keep raw coder:<instance>/<model> identities so catalog, routing, and policy checks validate the Coder gateway entry (cross-typed instances canonicalize by NAME to the wrong direct provider); stats resolve via getModelStatsResolved. Red-checked tests in suggestion.test.ts.
|
@codex review |
|
Codex Review: Didn't find any major issues. Bravo. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
1 participant
Summary
Generalizes the Coder AI Gateway integration from two hardcoded provider names (
anthropic,openai) to arbitrary server-side provider instances: dynamic provider discovery, type-based wire-protocol routing forcoder:<provider>/<model>strings, and a "Refresh models" action that re-discovers without a re-login.Background
Coder AI Gateway deployments configure N provider instances with arbitrary names and types (
openai,anthropic,azure,google,openai-compat,openrouter,vercel,bedrock,copilot), each mounted at/api/v2/aibridge/<name>/. Mux previously:coder:<x>/<model>string wherexwas not exactlyanthropicoropenai, so other providers were unroutable even manually;This made the integration borderline unusable on real multi-provider deployments.
Implementation
coderOauthService.ts): first triesGET /api/v2/ai/providers(authoritative{name, type, enabled}list). That endpoint requires site-wide AIProvider read, so on 403/404 it falls back to probing the default type-named routes plus any previously known instances (discoveredProviders∪ user-managedadditionalProviders). Catalog writes are per-provider conclusive:okreplaces that provider's entries, 404 clears them, and transient errors carry the previous entries forward — one provider whose upstream rejects/v1/models(e.g. AWS Bedrock behind the passthrough) can no longer poison the whole catalog write. Existing invariants are preserved: policy-unfiltered durable catalog, manual entries andremovedModelshonored, credential-pinned locked writes so stale discovery can't commit over a disconnect/re-login.providerModelFactory.ts):coder:<name>/<model>splits on the first slash; the name resolves throughadditionalProviders→discoveredProviders→ the name === type default. The resolved type picks the wire:anthropic/bedrock→ Anthropic messages,openai→ OpenAI Responses, other OpenAI-wire types → chat completions (compat upstreams don't reliably support/v1/responses).copilotis rejected with a clear error (needs client-minted request-time tokens)./v1/modelsis a gateway passthrough and Anthropic 400s withoutanthropic-version; discovery now sends it for Anthropic-wire instances (found live on dev.coder.com — without it, every Anthropic model was hidden).coderOauth.refreshModelsIPC + Settings "Refresh models" button + command palette action.discoveredProviders(discovery-written metadata, never a routing gate) andadditionalProviders(user escape hatch for custom-named instances on deployments where members cannot list providers) on the coder provider config.Known follow-ups (out of scope): canonical→gateway auto-routing (
anthropic:xvia Coder) still assumes default-named instances;additionalProvidershas no Settings UI yet (documented hand-edit).Validation
additionalProvidersprobing/routing, wire-protocol mapping (incl. bedrock→anthropic and copilot rejection), and the anthropic-version header split.prod-anthropic,llm-proxy/openai-compat), catalog + provider metadata persisted, chats routed through/aibridge/llm-proxy/v1/chat/completionsand/aibridge/prod-anthropic/v1/messages, and the Refresh button picking up server-side catalog growth without re-login.anthropic/openai/openai-compat(278 models); real completions verified on all three wires (coder:anthropic/claude-sonnet-4-5-20250929,coder:openai/gpt-5.2,coder:openai-compat/gpt-4.1-mini) with usage/cost tracking.Risks
coder:anthropic/.../coder:openai/...routable with no metadata at all, discovery invariants are covered by the pre-existing test suite (all green), and both wires were re-verified live.discoveredProvidersis additive config; older builds ignore it (upgrade↔downgrade safe).Generated with
mux• Model:anthropic:claude-fable-5• Thinking:xhigh• Cost:$92.30