Skip to content

fix(llm-api-gateway): enforce model uris allowlist on /v1/embeddings - #699

Open
along-2017 wants to merge 1 commit into
mainfrom
fix/llm-api-gateway/embeddings-uri-allowlist
Open

fix(llm-api-gateway): enforce model uris allowlist on /v1/embeddings#699
along-2017 wants to merge 1 commit into
mainfrom
fix/llm-api-gateway/embeddings-uri-allowlist

Conversation

@along-2017

@along-2017 along-2017 commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

TL;DR

Enforce the model uris allowlist on /v1/embeddings. A model that declares only /v1/chat/completions was refused on /v1/responses with 400 but still served /v1/embeddings with 200. Both undeclared endpoints now return the same 400.

Additional Details

Only the /v1/responses handler checked llmConfig.uris. Embeddings validation checks a static capability map that defaults to enabled for models not in the gateway static config, so undeclared embeddings requests were proxied through. This change generalizes the responses-only check into requireModelURI and calls it during embeddings validation. Models with no uris declared keep current behavior.

For the Reviewer

The requireModelURI refactor in responses_handler.go is behavior preserving; the only behavior change is the new check in validateEmbeddingRequest.

For QA

go test ./api/ passes, including the new regression test TestEmbeddingsRejectsModelWithoutEmbeddingsURI (400 for a model that does not declare /v1/embeddings) and the existing embeddings and responses tests. QA re-validation of the original three-endpoint scenario is recommended.

Issues

Fixes #ISSUE_NUMBER

Checklist

  • I am familiar with the Contributing Guidelines.
  • I have signed off my commits for Developer Certificate of Origin (DCO) compliance.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

Summary by CodeRabbit

  • Bug Fixes
    • Embedding requests are now rejected with a clear bad-request response when the selected model does not support the embeddings endpoint.
    • Endpoint validation is applied consistently across embedding and response requests.

@along-2017
along-2017 requested a review from a team as a code owner August 6, 2026 00:49
@along-2017
along-2017 requested a review from borao August 6, 2026 00:49
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: dc5f0d04-2097-4aed-92f1-2ead290b6fa5

📥 Commits

Reviewing files that changed from the base of the PR and between 3de9a59 and 8be1d92.

📒 Files selected for processing (3)
  • src/invocation-plane-services/llm-api-gateway/api/embeddings_handler_test.go
  • src/invocation-plane-services/llm-api-gateway/api/openai_endpoint_handlers.go
  • src/invocation-plane-services/llm-api-gateway/api/responses_handler.go
🚧 Files skipped from review as they are similar to previous changes (3)
  • src/invocation-plane-services/llm-api-gateway/api/responses_handler.go
  • src/invocation-plane-services/llm-api-gateway/api/openai_endpoint_handlers.go
  • src/invocation-plane-services/llm-api-gateway/api/embeddings_handler_test.go

📝 Walkthrough

Walkthrough

The gateway validates routed model support for endpoint-specific URIs. Embedding requests require /v1/embeddings. Responses validation uses the shared URI helper. Tests cover rejected and accepted embedding requests.

Changes

Embedding URI validation

Layer / File(s) Summary
Generalize model URI validation
src/invocation-plane-services/llm-api-gateway/api/responses_handler.go
Adds reusable validation for caller-provided endpoint paths and updates responses validation to use it.
Validate embedding endpoint support
src/invocation-plane-services/llm-api-gateway/api/openai_endpoint_handlers.go, src/invocation-plane-services/llm-api-gateway/api/embeddings_handler_test.go
Adds the /v1/embeddings path, validates model support, and tests rejected and accepted requests.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: famousdirector, max-nv

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title uses the required Conventional Commits format and accurately describes the embedding model URI allowlist fix.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/llm-api-gateway/embeddings-uri-allowlist

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.2)

level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies"


Comment @coderabbitai help to get the list of available commands.

@along-2017 along-2017 self-assigned this Aug 6, 2026
@along-2017
along-2017 removed the request for review from borao August 6, 2026 00:49
@along-2017
along-2017 force-pushed the fix/llm-api-gateway/embeddings-uri-allowlist branch from 624b386 to e8111f0 Compare August 6, 2026 01:34
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

The gateway rejected undeclared /v1/responses requests but served
/v1/embeddings for models that do not declare it in their uris list.
Generalize the responses-only check into requireModelURI and apply it
during embeddings validation, returning the same 400 as /v1/responses.

Signed-off-by: along <along@nvidia.com>
@along-2017
along-2017 force-pushed the fix/llm-api-gateway/embeddings-uri-allowlist branch from e8111f0 to 8be1d92 Compare August 6, 2026 01:48
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant