Skip to content

[Bug] Local OpenAI-compatible model: supported_reasoning_efforts from /v1/models ignored, no variants via /variants #10

Description

@cmoalem

Bug: Local OpenAI-compatible model — supported_reasoning_efforts from /v1/models is ignored, no variants available via /variants

Koder version: 0.4.1
Provider: local OpenAI-compatible server (ExLlamaV3 / TabbyAPI) at http://localhost:8888/v1, configured in koder.jsonc via @ai-sdk/openai-compatible
Model: qwen3.8-27b-exl3-4.0bpw
OS: Linux

The local server advertises reasoning efforts

GET http://localhost:8888/v1/models returns:

{
  "id": "qwen3.8-27b-exl3-4.0bpw",
  "object": "model",
  "owned_by": "exl3",
  "max_model_len": 262144,
  "supported_reasoning_efforts": ["low", "medium", "xhigh"]
}

Observed behavior

Koder does not process the reported reasoning-effort variants for local models:

  • /variants offers no variants for the local model, so the reasoning effort cannot be switched in-session.
  • Every assistant message for the local model is recorded without a variant field, while platform models in the same sessions record e.g. variant: "medium".

Workaround currently in use

Hard-configuring the effort in koder.jsonc:

"models": {
  "qwen3.8-27b-exl3-4.0bpw": {
    "options": { "reasoningEffort": "medium" }
  }
}

Expected behavior

Koder should read supported_reasoning_efforts from the OpenAI-compatible /v1/models response and expose those efforts as switchable variants via /variants, the same way it does for platform models.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions