Skip to content

fix(provider): support custom provider image modalities - #51298

Open
saiqulhaq-hh wants to merge 1 commit into
anomalyco:devfrom
saiqulhaq-hh:fix/issue-47480-vision-capability
Open

saiqulhaq-hh wants to merge 1 commit into
anomalyco:devfrom
saiqulhaq-hh:fix/issue-47480-vision-capability

Conversation

@saiqulhaq-hh

@saiqulhaq-hh saiqulhaq-hh commented Sep 25, 2026 •

Copy link
Copy Markdown

Issue for this PR

Closes #47480

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Custom-provider models can reach the message filter with capabilities.input in the modalities array form, such as ["text", "image"]. The filter only checked the object form (input.image), so it replaced valid image parts with a "model does not support image input" error before calling the provider.

This change accepts both capability representations at that boundary: array capabilities use includes(modality), while registry-derived object capabilities keep the existing keyed lookup. A regression test covers an image-capable custom-provider model using the array form.

How did you verify your code works?

  • bun test test/provider/transform.test.ts from packages/opencode (586 passed)
  • bun typecheck from packages/opencode
  • bun run test from packages/opencode (3612 passed, 22 skipped, 1 todo)
  • Prettier and oxlint on the two changed files

I also ran the patched CLI with bun dev . and used the Read tool on a local image with this sanitized custom-provider configuration:

{
  "provider": {
    "litellm": {
      "npm": "@ai-sdk/openai-compatible",
      "options": {
        "baseURL": "https://<litellm-host>/v1",
        "apiKey": "{env:LITELLM_API_KEY}"
      },
      "models": {
        "minimax/MiniMax-M3": {
          "name": "MiniMax M3",
          "attachment": true,
          "modalities": {
            "input": ["text", "image"]
          }
        }
      }
    }
  }
}

The model successfully parsed text from the image instead of receiving the unsupported-image error.

Screenshots / recordings

Local test result: bun dev .

image

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

If you do not follow this template your PR will be automatically rejected.

@github-actions

Copy link
Copy Markdown
Contributor

The following comment was made by an LLM, it may be inaccurate:

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants