Skip to content

fix(model): register discovered endpoints with /v1 api_base#745

Closed
bussyjd wants to merge 2 commits into
mainfrom
fix/discovery-api-base-v1
Closed

fix(model): register discovered endpoints with /v1 api_base#745
bussyjd wants to merge 2 commits into
mainfrom
fix/discovery-api-base-v1

Conversation

@bussyjd

@bussyjd bussyjd commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Local-server auto-discovery (buildDiscoveredProvider, since v0.10.0 69f25bf) registers OpenAI-compatible endpoints with a bare api_base. LiteLLM's OpenAI provider does not append /v1 (CLAUDE.md pitfall 6), so requests through the discovered entry hit POST /chat/completions → vLLM 404 {'detail':'Not Found'}. When an explicit obol model setup custom --endpoint .../v1 entry shares the model group, LiteLLM shuffles ~50/50 between the good and poisoned deployment.

Evidence (v0.13.0-rc4 release-smoke on spark1): vLLM access log 06:12:15 POST /chat/completions 404 → retry 06:12:17 POST /v1/chat/completions 200 (flow-03 step[2] flake), and 06:25:38 POST /chat/completions 404 matching Alice's verifier handler returned 404, skipping settlement (flow-11 step[43] hard fail — the only red step in an otherwise green run). Reproduced by curl: /v1/chat/completions → 200, /chat/completions → 404.

Not an rc4 regression — latent since v0.10.0, exposed on QA hosts where vLLM sits on the discovery-probed port with served-model-name == OBOL_LLM_MODEL. Blocks a green release-smoke, so it should ride in v0.13.0.

Companion: #744 (flow-03 retry + surface HTTP status).

https://claude.ai/code/session_014YjPMViNrZ7zBVgUQzwEKk

Local-server auto-discovery registered OpenAI-compatible endpoints
(vLLM on :8000) with a bare api_base. LiteLLM's OpenAI provider does
not append /v1 (CLAUDE.md pitfall 6), so every request through the
discovered entry hit POST /chat/completions and 404'd. When an explicit
'obol model setup custom --endpoint .../v1' entry shared the same model
group, requests coin-flipped between the good and bad deployment —
root cause of the intermittent release-smoke flow-11 step[43] 404 and
the flow-03 step[2] flake (latent since v0.10.0, 69f25bf).

Claude-Session: https://claude.ai/code/session_014YjPMViNrZ7zBVgUQzwEKk
@bussyjd
bussyjd force-pushed the fix/discovery-api-base-v1 branch from 78c2880 to 171731b Compare July 14, 2026 07:03
@bussyjd

bussyjd commented Jul 14, 2026

Copy link
Copy Markdown
Contributor Author

Full /v1 pipeline audit (requested before merge)

Every producer and consumer of OpenAI-compatible endpoint URLs in the repo, verified against source:

# Site Convention Verdict
1 obol model setup customAddCustomEndpointWithOptions (model.go:995) endpoint must include /v1; enforced by validation step 2, which probes <endpoint>/chat/completions — a /v1-less vLLM endpoint fails the inference probe and is rejected at setup time ✅ correct, self-protecting
2 auto-discovery → buildDiscoveredProvider (discover.go) was the only violator: wrote bare api_base. Discovery's own gate (fetchModels, detect.go:282) requires GET <base>/v1/models → 200, so every endpoint it can emit provably serves under /v1 — the +"/v1" append is safe by construction for all discoverable server types (vLLM, llama.cpp, LM Studio) ✅ fixed by this PR
3 shared chokepoint buildCustomEndpointEntryWithOptions (model.go) writes api_base verbatim; both callers now uphold the invariant ✅ invariant now documented on the builder
4 WarnAndStripV1Suffix (model.go:1699) dead code (zero callers) whose doc claimed the opposite ("LiteLLM auto-appends /v1") — the fossil that kept re-seeding this confusion ✅ deleted in this PR
5 llm.yaml paid/* default route api_base: http://x402-buyer...:8402/v1 with /v1, pitfall documented inline (ffa7cf4 fixed this same class) ✅ correct
6 per-purchase entries → buyerAPIBase() (purchase_helpers.go:281) http://x402-buyer.<ns>...:8402/v1 — with /v1, plus legacy-base migration ✅ correct
7 buyer-side URL building (buyprompts.ChatCompletionsURL, buy/discover.go:567) opposite convention — base without /v1, appends /v1/chat/completions itself; internally consistent and documented as lockstep with the verifier ✅ correct (different but coherent convention)
8 seller-side verifier normalizeChatCompletionsPath (verifier.go:857) tolerantly rewrites "", /v1, /chat/completions/v1/chat/completions for inference/agent offers ✅ correct
9 hermes agent renderer + openclaw hardcoded http://litellm...:4000/v1 ✅ correct
10 Ollama path separate ollama/ provider (native API, no /v1 semantics) — explicitly excluded from discovery (shouldRegisterEndpoint) ✅ not applicable

Root contract, now written on the chokepoint: LiteLLM's openai/ provider sends to <api_base>/chat/completions verbatim and never appends /v1 — so every LiteLLM api_base must carry /v1, while buyer-tooling bases must not (they append the full path).

Known residual gap (out of scope, tracked): the config-drift checker compares model names only, so a same-name entry with a divergent api_base (exactly this incident's poisoned group) is invisible to it.

bussyjd added a commit that referenced this pull request Jul 14, 2026
- pitfall 6 generalized: EVERY LiteLLM openai/ api_base must include
  /v1 (LiteLLM posts <api_base>/chat/completions verbatim); buyer-side
  tooling uses the opposite convention; discovery fixed in #745
- new pitfall 22: poisoned model group (two deployments, one bare
  api_base -> intermittent 404, no retry; GET /model/info vs CM is
  the decisive check)
- new pitfall 23: buyer disconnect != no charge (hermes non-streaming
  path completes + settles after abort; keep runs short via
  spec.maxTurns; maxConcurrentRuns semantics; 4xx never settled)
- skill: lessons table updated (poisoned model group, parserless QA
  endpoint -> relaunch with publisher-exact launcher), version 3.2.0

Claude-Session: https://claude.ai/code/session_014YjPMViNrZ7zBVgUQzwEKk
@bussyjd

bussyjd commented Jul 14, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by #749 (integration/v0.13.0-rc4) — this branch is merged into the integration branch verbatim (merge commits a02b62f/7d963125) and ships there. Validated by the fully green release smoke rc4-run7-20260714-171005 (18/18 flows incl. flow-13/14, on-chain receipts in #749).

@bussyjd bussyjd closed this Jul 14, 2026
auto-merge was automatically disabled July 14, 2026 10:50

Pull request was closed

OisinKyne pushed a commit that referenced this pull request Jul 14, 2026
Zero callers anywhere (including tests) — flagged by the /v1 pipeline
audit's dead-code sweep alongside WarnAndStripV1Suffix (removed in
#745). Other deadcode-tool hits in internal/{buy,inference} are
test-covered exported API and were left alone.

Claude-Session: https://claude.ai/code/session_014YjPMViNrZ7zBVgUQzwEKk
OisinKyne pushed a commit that referenced this pull request Jul 14, 2026
- pitfall 6 generalized: EVERY LiteLLM openai/ api_base must include
  /v1 (LiteLLM posts <api_base>/chat/completions verbatim); buyer-side
  tooling uses the opposite convention; discovery fixed in #745
- new pitfall 22: poisoned model group (two deployments, one bare
  api_base -> intermittent 404, no retry; GET /model/info vs CM is
  the decisive check)
- new pitfall 23: buyer disconnect != no charge (hermes non-streaming
  path completes + settles after abort; keep runs short via
  spec.maxTurns; maxConcurrentRuns semantics; 4xx never settled)
- skill: lessons table updated (poisoned model group, parserless QA
  endpoint -> relaunch with publisher-exact launcher), version 3.2.0

Claude-Session: https://claude.ai/code/session_014YjPMViNrZ7zBVgUQzwEKk
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant