fix(llm-api-gateway): enforce model uris allowlist on /v1/embeddings - #699
fix(llm-api-gateway): enforce model uris allowlist on /v1/embeddings#699along-2017 wants to merge 1 commit into
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (3)
🚧 Files skipped from review as they are similar to previous changes (3)
📝 WalkthroughWalkthroughThe gateway validates routed model support for endpoint-specific URIs. Embedding requests require ChangesEmbedding URI validation
Estimated code review effort: 2 (Simple) | ~10 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 golangci-lint (2.12.2)level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies" Comment |
624b386 to
e8111f0
Compare
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
The gateway rejected undeclared /v1/responses requests but served /v1/embeddings for models that do not declare it in their uris list. Generalize the responses-only check into requireModelURI and apply it during embeddings validation, returning the same 400 as /v1/responses. Signed-off-by: along <along@nvidia.com>
e8111f0 to
8be1d92
Compare
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
TL;DR
Enforce the model uris allowlist on /v1/embeddings. A model that declares only /v1/chat/completions was refused on /v1/responses with 400 but still served /v1/embeddings with 200. Both undeclared endpoints now return the same 400.
Additional Details
Only the /v1/responses handler checked llmConfig.uris. Embeddings validation checks a static capability map that defaults to enabled for models not in the gateway static config, so undeclared embeddings requests were proxied through. This change generalizes the responses-only check into requireModelURI and calls it during embeddings validation. Models with no uris declared keep current behavior.
For the Reviewer
The requireModelURI refactor in responses_handler.go is behavior preserving; the only behavior change is the new check in validateEmbeddingRequest.
For QA
go test ./api/ passes, including the new regression test TestEmbeddingsRejectsModelWithoutEmbeddingsURI (400 for a model that does not declare /v1/embeddings) and the existing embeddings and responses tests. QA re-validation of the original three-endpoint scenario is recommended.
Issues
Fixes #ISSUE_NUMBER
Checklist
Summary by CodeRabbit