fix(model): register discovered endpoints with /v1 api_base#745
Closed
bussyjd wants to merge 2 commits into
Closed
fix(model): register discovered endpoints with /v1 api_base#745bussyjd wants to merge 2 commits into
bussyjd wants to merge 2 commits into
Conversation
Local-server auto-discovery registered OpenAI-compatible endpoints (vLLM on :8000) with a bare api_base. LiteLLM's OpenAI provider does not append /v1 (CLAUDE.md pitfall 6), so every request through the discovered entry hit POST /chat/completions and 404'd. When an explicit 'obol model setup custom --endpoint .../v1' entry shared the same model group, requests coin-flipped between the good and bad deployment — root cause of the intermittent release-smoke flow-11 step[43] 404 and the flow-03 step[2] flake (latent since v0.10.0, 69f25bf). Claude-Session: https://claude.ai/code/session_014YjPMViNrZ7zBVgUQzwEKk
bussyjd
force-pushed
the
fix/discovery-api-base-v1
branch
from
July 14, 2026 07:03
78c2880 to
171731b
Compare
Contributor
Author
Full
|
| # | Site | Convention | Verdict |
|---|---|---|---|
| 1 | obol model setup custom → AddCustomEndpointWithOptions (model.go:995) |
endpoint must include /v1; enforced by validation step 2, which probes <endpoint>/chat/completions — a /v1-less vLLM endpoint fails the inference probe and is rejected at setup time |
✅ correct, self-protecting |
| 2 | auto-discovery → buildDiscoveredProvider (discover.go) |
was the only violator: wrote bare api_base. Discovery's own gate (fetchModels, detect.go:282) requires GET <base>/v1/models → 200, so every endpoint it can emit provably serves under /v1 — the +"/v1" append is safe by construction for all discoverable server types (vLLM, llama.cpp, LM Studio) |
✅ fixed by this PR |
| 3 | shared chokepoint buildCustomEndpointEntryWithOptions (model.go) |
writes api_base verbatim; both callers now uphold the invariant |
✅ invariant now documented on the builder |
| 4 | WarnAndStripV1Suffix (model.go:1699) |
dead code (zero callers) whose doc claimed the opposite ("LiteLLM auto-appends /v1") — the fossil that kept re-seeding this confusion | ✅ deleted in this PR |
| 5 | llm.yaml paid/* default route |
api_base: http://x402-buyer...:8402/v1 with /v1, pitfall documented inline (ffa7cf4 fixed this same class) |
✅ correct |
| 6 | per-purchase entries → buyerAPIBase() (purchase_helpers.go:281) |
http://x402-buyer.<ns>...:8402/v1 — with /v1, plus legacy-base migration |
✅ correct |
| 7 | buyer-side URL building (buyprompts.ChatCompletionsURL, buy/discover.go:567) |
opposite convention — base without /v1, appends /v1/chat/completions itself; internally consistent and documented as lockstep with the verifier |
✅ correct (different but coherent convention) |
| 8 | seller-side verifier normalizeChatCompletionsPath (verifier.go:857) |
tolerantly rewrites "", /v1, /chat/completions → /v1/chat/completions for inference/agent offers |
✅ correct |
| 9 | hermes agent renderer + openclaw | hardcoded http://litellm...:4000/v1 |
✅ correct |
| 10 | Ollama path | separate ollama/ provider (native API, no /v1 semantics) — explicitly excluded from discovery (shouldRegisterEndpoint) |
✅ not applicable |
Root contract, now written on the chokepoint: LiteLLM's openai/ provider sends to <api_base>/chat/completions verbatim and never appends /v1 — so every LiteLLM api_base must carry /v1, while buyer-tooling bases must not (they append the full path).
Known residual gap (out of scope, tracked): the config-drift checker compares model names only, so a same-name entry with a divergent api_base (exactly this incident's poisoned group) is invisible to it.
bussyjd
enabled auto-merge (squash)
July 14, 2026 07:15
This was referenced Jul 14, 2026
bussyjd
added a commit
that referenced
this pull request
Jul 14, 2026
- pitfall 6 generalized: EVERY LiteLLM openai/ api_base must include /v1 (LiteLLM posts <api_base>/chat/completions verbatim); buyer-side tooling uses the opposite convention; discovery fixed in #745 - new pitfall 22: poisoned model group (two deployments, one bare api_base -> intermittent 404, no retry; GET /model/info vs CM is the decisive check) - new pitfall 23: buyer disconnect != no charge (hermes non-streaming path completes + settles after abort; keep runs short via spec.maxTurns; maxConcurrentRuns semantics; 4xx never settled) - skill: lessons table updated (poisoned model group, parserless QA endpoint -> relaunch with publisher-exact launcher), version 3.2.0 Claude-Session: https://claude.ai/code/session_014YjPMViNrZ7zBVgUQzwEKk
4 tasks
Contributor
Author
auto-merge was automatically disabled
July 14, 2026 10:50
Pull request was closed
OisinKyne
pushed a commit
that referenced
this pull request
Jul 14, 2026
Zero callers anywhere (including tests) — flagged by the /v1 pipeline audit's dead-code sweep alongside WarnAndStripV1Suffix (removed in #745). Other deadcode-tool hits in internal/{buy,inference} are test-covered exported API and were left alone. Claude-Session: https://claude.ai/code/session_014YjPMViNrZ7zBVgUQzwEKk
OisinKyne
pushed a commit
that referenced
this pull request
Jul 14, 2026
- pitfall 6 generalized: EVERY LiteLLM openai/ api_base must include /v1 (LiteLLM posts <api_base>/chat/completions verbatim); buyer-side tooling uses the opposite convention; discovery fixed in #745 - new pitfall 22: poisoned model group (two deployments, one bare api_base -> intermittent 404, no retry; GET /model/info vs CM is the decisive check) - new pitfall 23: buyer disconnect != no charge (hermes non-streaming path completes + settles after abort; keep runs short via spec.maxTurns; maxConcurrentRuns semantics; 4xx never settled) - skill: lessons table updated (poisoned model group, parserless QA endpoint -> relaunch with publisher-exact launcher), version 3.2.0 Claude-Session: https://claude.ai/code/session_014YjPMViNrZ7zBVgUQzwEKk
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Local-server auto-discovery (
buildDiscoveredProvider, since v0.10.0 69f25bf) registers OpenAI-compatible endpoints with a bareapi_base. LiteLLM's OpenAI provider does not append/v1(CLAUDE.md pitfall 6), so requests through the discovered entry hitPOST /chat/completions→ vLLM 404{'detail':'Not Found'}. When an explicitobol model setup custom --endpoint .../v1entry shares the model group, LiteLLM shuffles ~50/50 between the good and poisoned deployment.Evidence (v0.13.0-rc4 release-smoke on spark1): vLLM access log
06:12:15 POST /chat/completions 404→ retry06:12:17 POST /v1/chat/completions 200(flow-03 step[2] flake), and06:25:38 POST /chat/completions 404matching Alice's verifierhandler returned 404, skipping settlement(flow-11 step[43] hard fail — the only red step in an otherwise green run). Reproduced by curl:/v1/chat/completions→ 200,/chat/completions→ 404.Not an rc4 regression — latent since v0.10.0, exposed on QA hosts where vLLM sits on the discovery-probed port with served-model-name == OBOL_LLM_MODEL. Blocks a green release-smoke, so it should ride in v0.13.0.
Companion: #744 (flow-03 retry + surface HTTP status).
https://claude.ai/code/session_014YjPMViNrZ7zBVgUQzwEKk