feat(providers): add OrcaRouter provider - #361
Conversation
Add a named orcarouter provider for the OrcaRouter model-routing gateway (https://api.orcarouter.ai/v1, auth via ORCAROUTER_API_KEY), mirroring the existing openai provider shape. Registers it in the SKILLSPECTOR_PROVIDER selector, ships bundled token-budget metadata for the orcarouter/auto default, and documents it in README.md and .env.example. Signed-off-by: XiaoHuo888 <sjh00112233@outlook.com>
| class OrcaRouterProvider: | ||
| """OrcaRouter credentials + bundled-YAML metadata provider.""" | ||
|
|
||
| DEFAULT_MODEL = "orcarouter/auto" |
There was a problem hiding this comment.
[P1] orcarouter/auto is not a safe default for the SkillSpector structured-output path. LLMAnalyzerBase calls ChatOpenAI.with_structured_output(...), whose current default uses response_format: json_schema, but OrcaRouter documents that this is unsupported when the upstream is Anthropic and only works where the selected upstream can honor it. The seeded auto router matches every accessible chat model, so it may select such an upstream and make core analyzer calls fail even though the raw-chat smoke test succeeds. Please either use a concrete default model that guarantees JSON-schema output or a capability-constrained router, and add an OrcaRouter live structured-output test alongside tests/provider/test_provider_endpoint.py. References: structured outputs and auto router.
|
|
||
| models: | ||
| "orcarouter/auto": | ||
| context_length: 200000 |
There was a problem hiding this comment.
[P2] A fixed 200k/64k budget cannot describe orcarouter/auto, whose candidate backend changes per request and whose default allowed-model pattern matches every chat model the account can access. These values make SkillSpector accept roughly 150k input tokens and request up to 50k output tokens, which can exceed a selected backend context or output cap and turn larger scans into provider errors. Please use limits OrcaRouter contractually guarantees for every auto-router candidate, or avoid publishing metadata for this dynamic alias and require a concrete/custom-router registry entry. Reference: auto router.
2 participants
Summary
OrcaRouter is an OpenAI-compatible model routing gateway that brings 150+ models from OpenAI, Anthropic, Google, DeepSeek, Qwen, MiniMax and xAI behind a single endpoint and API key. Beyond routing, it runs gateway-level, zero-trust security for AI agents on the same endpoint - screening every prompt/response and governing every tool call on a default-deny basis, with no application code changes. This PR registers it as a named provider so users can opt in directly.
I'm an engineer on the OrcaRouter team.
What this changes
src/skillspector/providers/orcarouter/provider.py: newOrcaRouterProvider, mirroring the existingopenaiprovider shape (base URLhttps://api.orcarouter.ai/v1, auth viaORCAROUTER_API_KEY, bundled default modelorcarouter/auto)src/skillspector/providers/orcarouter/model_registry.yaml: token-budget metadata fororcarouter/auto, alongside the other providers' bundled registriessrc/skillspector/providers/orcarouter/__init__.py: package exportssrc/skillspector/providers/__init__.py: registerorcarouterin theSKILLSPECTOR_PROVIDERselector and docstests/unit/test_providers.py: unit tests for credentials, chat-model construction, metadata lookup and provider selectionREADME.md/.env.example: provider table row, setup snippet and env-var referenceWhy a named provider rather than the OpenAI-compatible escape hatch
SkillSpector's
openaiprovider already accepts anyOPENAI_BASE_URL, which would work with OrcaRouter. A dedicatedorcarouterprovider keeps the provider table honest and discoverable, matching how the other hosted providers (anthropic,nv_build, ...) each own their credential env var and bundled default model. Named routers such asorcarouter/autopick an upstream per request, so users get a working default without hunting for a model id.How to use it
sk-orca-).SKILLSPECTOR_PROVIDER=orcarouterandORCAROUTER_API_KEY.skillspector scan ./my-skill/. The default model isorcarouter/auto; override withSKILLSPECTOR_MODELif needed.Verification
pytest tests/unit/test_providers.py-> 86 passed, 9 skipped (7 new OrcaRouter tests). Full unit suite: 965 passed, 12 skipped. The 5 failures intests/unit/test_create_github_release.pyreproduce on the clean baseline (environmental, unrelated to this change).ruff checkandruff format --checkclean on all changed files.SKILLSPECTOR_PROVIDER=orcarouterand a real key,create_chat_model("orcarouter/auto")returnedOK(finish_reasonstop) with usage metadata fromhttps://api.orcarouter.ai/v1.git commit -s).