Skip to content

fix(vision): bind generated capabilities to provider transport - #581

Open
luvs01 wants to merge 9 commits into
devfrom
codex/propose-fix-for-vision-sidecar-vulnerability
Open

luvs01 wants to merge 9 commits into
devfrom
codex/propose-fix-for-vision-sidecar-vulnerability

Conversation

@luvs01

@luvs01 luvs01 commented Sep 21, 2026

Copy link
Copy Markdown
Owner

Summary

  • A runtime change began consulting generated vendor capability metadata by provider name alone, which could misclassify same-named custom providers and cause images or adjacent text to be sent to an unrelated vision sidecar or stripped before reaching the intended destination.
  • src/vision/eligibility.ts now returns undefined (unknown) unless providerMatchesRegistryTransportOrAlias proves the configured transport belongs to the registry row that owns the name.
  • src/providers/registry-transport.ts centralizes that check: generated-metadata aliases (geminigoogle, anthropic-keyanthropic-apikey, gemini-vertex, antigravity/gemini-antigravity) and case-varied registry ids resolve to the declaring entry — resolveMetadataProvider binds those names to the owner's bundle, so an id-only lookup would drop verdicts the registry still owns.
  • Every name routing does not pin — alias/case-varied names, allowBaseUrlOverride presets, and preserveCustomDestination presets — is validated against the owner's declared destinations: the configured adapter, auth mode, and normalized endpoint must literally equal the fixed transport, a documented baseUrlChoices endpoint (a "custom" choice declares no URL and cannot match), or a destinationAliases former endpoint on that destination's own adapter. Routing binds transports by case-sensitive exact entry.id === providerName, so the owner's pinning rule would otherwise apply vendor verdicts to destinations routing still serves as custom — e.g. a moonshot row pointed at an arbitrary gateway, or a custom gemini/Gemini endpoint.
  • metadataImageInput folds the provider name case-insensitively (bundle keys are all lowercase), so a case-varied configured name such as ZAI reaches its owner's bundle once the transport check passes.
  • Same-name custom destinations keep the legacy unknown-capability semantics: images are not redirected or stripped when the transport differs from the registry row.
  • src/providers/registry.ts stays inside its file-size-ratchet cap by placing the matcher in the dedicated registry-transport.ts module and exporting normalizedProviderEndpoint.
  • structure/runtime.md documents that vendor metadata applies only at a declared destination (fixed, baseUrlChoices, or destinationAliases), because routing binds transports by exact registry id only.

Verification

  • ./node_modules/.bin/bun test tests/vision/vision-eligibility.test.ts — 29 tests, 0 failures (repo-pinned Bun 1.4.0; covers a zhipu-bigmodel-responses custom preset, alias/case-varied gemini/Gemini/GOOGLE destinations, moonshot at canonical/declared baseUrlChoices/arbitrary gateway, and ZAI on the zai destinationAliases former Chat endpoint keeping glm-5.3 text-only).
  • ./node_modules/.bin/bun test tests/providers/provider-registry-parity.test.ts tests/server/config.test.ts tests/providers/provider-model-discovery-contract.test.ts — 323 tests, 0 failures.
  • ./node_modules/.bin/bun run typecheck, bun run structure:check, bun run privacy:scan, bun scripts/file-size-ratchet.ts — all clean.
  • End-to-end: management API + dashboard verify text-only models excluded under gemini/GOOGLE/google at canonical transport and preserved at custom endpoints.

Checklist

  • Scope stays focused and avoids unrelated cleanup.
  • Docs or release notes were updated when needed.
  • Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults.

Link to Devin session: https://app.devin.ai/sessions/ac686e5ea38b4a64940ec31393b906de
Open in Devin Desktop: https://app.devin.ai/desktop/session/ac686e5ea38b4a64940ec31393b906de?variant=devin
Requested by: @luvs01


Devin Review

devin-ai-integration Bot and others added 2 commits September 20, 2026 13:22
…forwarded role (lidge-jun#5334 follow-up)

lidge-jun#5334 made the developer wire role tri-state: an undeclared destination folds
it to system. Two suites asserting role:"developer" on the Chat wire were
missed because they are about tool-result repair ordering and document parts,
not role selection — declare the destination, per the convention the change
established.

Verified: both files fail on dev@600075d2 with system-for-developer wire roles
and pass with the declaration.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Sep 21, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Repository: luvs01/opencodex/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 9924f7db-d5f2-49c9-84e8-4196a43bb533


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 21, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-21T00:32:34.246145Z a2aadb7 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

Copy link
Copy Markdown

Deterministic PR hygiene checks passed.

@github-actions github-actions Bot added the bug Something isn't working label Sep 21, 2026
devin-ai-integration[bot]

This comment was marked as resolved.

chatgpt-codex-connector[bot]

This comment was marked as resolved.

…etadata alias

The transport guard added in a2aadb7 looked the provider name up as a
registry id only, so a provider saved under a canonical metadata alias
(gemini, anthropic-key, gemini-vertex, ...) failed the lookup and its
vendor metadata verdict was discarded — generated text-only models fell
back to unknown, bypassing vision preprocessing and picker exclusion.

providerMatchesRegistryTransportOrAlias resolves the name to the entry
that owns it (by id or declared extraMetadataAliases, case-folded like
resolveMetadataProvider) and validates the configured transport against
that entry, so an aliased row follows its owner's pinning rule.

Co-Authored-By: Epinephrine <luvs01@hanmail.net>
@devin-ai-integration

Copy link
Copy Markdown

Fixed in 3cb3d0a. Added providerMatchesRegistryTransportOrAlias in src/providers/registry.ts: resolves a configured name to its owning registry row — by id, or by extraMetadataAliases (geminigoogle, anthropic-keyanthropic, gemini-vertex, antigravity/gemini-antigravity) case-folded the way resolveMetadataProvider accepts saved keys — then validates the configured adapter/endpoint against that row. The eligibility guard now uses it, so an alias-named provider on the canonical transport keeps its metadata verdict (text-only gemini-live-2.5-flash-preview-native-audiofalse, sidecar required), while a same-named custom destination still returns undefined. Regression tests 11e (alias + text-only model, incl. case-folded Gemini) and 11f (alias-named custom destination parity) added in tests/vision/vision-eligibility.test.ts.

@devin-ai-integration

Copy link
Copy Markdown

Fixed in 3cb3d0a — same root cause. The guard now calls providerMatchesRegistryTransportOrAlias, which resolves metadata aliases to the registry entry that declares them (geminigoogle, etc.) before comparing transport, so the alias keeps the owner's verdict instead of falling to undefined. Covered by new tests 11e/11f (canonical alias with text-only generated model; same-named custom destination).

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration Bot and others added 3 commits September 22, 2026 05:33
Routing binds a name to a registry transport by exact id only, so an alias- or case-named row keeps its configured destination. The owning entry's generated verdict is now authoritative only when the configured adapter, auth mode, and normalized endpoint literally equal that entry's declared transport; the previous fallback applied the owner's pinning rule (google is name-pinned, so it returned true unconditionally) and let a custom gemini/anthropic-key/case-varied destination inherit verdicts for an upstream it never reaches.

Co-Authored-By: Epinephrine <luvs01@hanmail.net>
…ility

Co-Authored-By: Epinephrine <luvs01@hanmail.net>
…stry.ts

The alias/case-varied transport check grew src/providers/registry.ts to 280
lines, over its file-size-ratchet cap of 232. Move the matcher to a
dedicated module and export normalizedProviderEndpoint; behavior unchanged.

Co-Authored-By: Epinephrine <luvs01@hanmail.net>
@devin-ai-integration

Copy link
Copy Markdown

End-to-end verified on Windows (bun start, dashboard at localhost:10100).

Seeded config.json with four google-adapter providers to exercise every name-resolution path the fix touches: gemini (metadata alias), Gemini (case-varied alias, custom endpoint), GOOGLE (case-varied id), google (exact id) — plus custom-model rows for a text-only model (gemini-live-2.5-flash-preview-native-audio) and an image-capable one (gemini-2.5-flash).

Vision-sidecar picker (GET /api/sidecar-settings → dashboard dropdown): the text-only model is excluded under gemini, GOOGLE, and google at the canonical endpoint — vendor metadata now applies to alias/case-named providers whose transport literally matches. The same model stays eligible under Gemini pointing at a custom gateway (capability unknown → image boundary preserved). Both behaviors in one screenshot:

Vision sidecar dropdown — alias filtering
Vision sidecar dropdown — google/GOOGLE rows

Write gate + regression details
  • PUT /api/sidecar-settings with the text-only model under gemini400 "cannot describe images"; under Gemini (custom endpoint) → 200 (persisted as the selected describer shown in the screenshot).
  • Regression: Models/Providers pages render all alias/case-named provider sections without crashes or console-visible errors.
  • Not covered: requiresVisionPreprocessing request-path behavior (needs live upstream credentials to dispatch a real image request).

@devin-ai-integration

Copy link
Copy Markdown

Fixed in c1d7a58 (extracted to src/providers/registry-transport.ts in a449274). For names that are not exact registry ids, the matcher now requires the configured adapter, auth mode, and normalized endpoint to literally equal the resolved alias owner's declared transport — routing binds transports by exact case-sensitive entry.id === providerName lookup only, so the owner's pinning rule is no longer reused. Regression coverage added for a custom alias-named destination (gemini on a custom gateway and on a non-matching adapter), a case-varied custom provider (Gemini), and a forward-auth alias at the canonical endpoint (tests/vision/vision-eligibility.test.ts 11f/11g/11h).

@devin-ai-integration

Copy link
Copy Markdown

Addressed — the description now documents the alias/case-varied ownership semantics explicitly: generated-metadata aliases and case-varied ids resolve to the declaring registry entry, and for non-exact names the configured adapter, auth mode, and normalized endpoint must literally equal that entry's declared transport rather than reusing the owner's pinning rule.

devin-ai-integration[bot]

This comment was marked as resolved.

…ry endpoints

providerMatchesRegistryTransportOrAlias returned true unconditionally for
exact ids on presets whose URL routing honors (allowBaseUrlOverride) and
dropped verdicts for names at declared former endpoints. Both paths now
share declared-destination validation: the configured adapter, auth mode,
and normalized endpoint must literally equal the fixed transport, a
documented baseUrlChoices endpoint, or a destinationAliases former
endpoint; arbitrary custom endpoints are unknown. metadataImageInput
folds the provider name case-insensitively so bundle keys (all lowercase)
resolve for case-varied configured names like ZAI.

Co-Authored-By: Epinephrine <luvs01@hanmail.net>
@devin-ai-integration

Copy link
Copy Markdown

Resolved — fixed in 82ef87e.

1 similar comment
@devin-ai-integration

Copy link
Copy Markdown

Resolved — fixed in 82ef87e.

@devin-ai-integration

Copy link
Copy Markdown

Resolved — consolidation landed in 82ef87e.

devin-ai-integration[bot]

This comment was marked as resolved.

For an exact allowBaseUrlOverride id, routedProviderConfig canonicalizes
the adapter to entry.adapter and derives the auth mode — only the
configured URL reaches the wire — so literal adapter/auth matching
dropped valid verdicts from canonical rows missing fields like authMode
(a legacy google-antigravity row on the declared Google transport became
unknown and skipped preprocessing). Override ids now prove ownership by
endpoint membership in the entry's declared destinations, while
preserveCustomDestination rows — never canonicalized — keep literal
adapter/auth/endpoint matching, and the alias/case path is unchanged.

Co-Authored-By: Epinephrine <luvs01@hanmail.net>
@devin-ai-integration

Copy link
Copy Markdown

Resolved — fixed in 2fc1892.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

aardvark bug Something isn't working codex

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant