Skip to content

Paginate the model-provider-service listing, and resolve by name as a fallback - #264

Merged
tt-le merged 1 commit into
mainfrom
tien/mps-listing-pagination
Aug 5, 2026
Merged

Paginate the model-provider-service listing, and resolve by name as a fallback#264
tt-le merged 1 commit into
mainfrom
tien/mps-listing-pagination

Conversation

@tt-le

@tt-le tt-le commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Problem

main.tien_le.openai_all exists — the UC explore page shows it, and a direct GET returns HTTP 200 — but ucode configure reported:

Model provider service 'main.tien_le.openai_all' was not found.

resolve_provider_service lists the workspace's provider services and filters in memory, and list_model_provider_services made a single unpaginated request, discarding next_page_token. The metastore in question has 30 services across 2 pages, so ucode saw an arbitrary 16 of them. Anything on a later page was invisible — to the resolver, and to the interactive "Select a model provider service" picker.

GET  .../model-provider-services/main.tien_le.openai_all   → HTTP 200  (exists)
LIST as ucode did it                                        → 16 of 30, absent
LIST paginated                                              → page 2, found

Changes

  • list_model_provider_services paginates, mirroring list_model_services directly above it: bounded page_size, follows next_page_token, and a seen_tokens guard in case a server echoes one back. A mid-pagination failure degrades to partial results rather than an error, since partial data beats none.
  • Optional parent to scope the listing to one catalog.schema. The proto notes the metastore-wide default is an internal scope that is "likely to be deprecated", so callers that know the schema should pass it. The picker still needs the broad list, so that path keeps the metastore scope — now paginated.
  • get_model_provider_service fetches one service by name, and resolve_provider_service falls back to it before concluding "not found". A named service shouldn't be judged absent on the strength of a listing that might be incomplete; only a 404 there means it really is. The list stays first, because the picker needs to enumerate and the "Available for <tool>: ..." hint is built from it.
  • Entry parsing extracted to _provider_service_entry, shared by both paths.

Test plan

  • Tests: +8 covering token-following, the repeated-token guard, partial results on a mid-pagination failure, the error path when nothing was collected, parent scoping, page_size always being sent, and the by-name GET (success + 404).
  • test_follows_next_page_token was checked against the old single-page behavior, where it fails with Right contains one more item: 'main.s.two' — the same page-2 loss that caused the report, rather than merely asserting the new code's output.
  • Manual, against eng-ml-inference.staging: the listing goes from 16 to 30 services; main.tien_le.openai_all now appears; parent="main.tien_le" returns exactly the three services in that schema; resolving a genuinely absent name still errors with the available-names hint.
  • Full suite green apart from test_e2e_user_agent.py::test_user_agent_arrives_at_gateway, which fails identically on unmodified main (it needs a live gateway).

This pull request and its description were written by Isaac.

@tt-le
tt-le requested review from asujithan and masonc08 August 5, 2026 15:37
`main.tien_le.openai_all` exists — the UC explore page shows it, and a direct GET
returns HTTP 200 — but ucode reported:

    Model provider service 'main.tien_le.openai_all' was not found.

`resolve_provider_service` lists and filters in memory, and the listing made a
single unpaginated request. That metastore has 30 services across 2 pages, so
ucode saw an arbitrary 16 and discarded the `next_page_token`. Anything on a later
page was invisible — to the resolver and to the interactive picker alike.

- `list_model_provider_services` now pages, mirroring `list_model_services` right
  above it (bounded `page_size`, `next_page_token`, a `seen_tokens` guard against a
  server echoing one back). A mid-pagination failure degrades to partial results
  rather than an error, since partial data is more useful than none.
- It also accepts `parent` to scope the listing to one `catalog.schema`. The
  metastore-wide default is documented in the proto as an internal scope that is
  "likely to be deprecated", so callers that know the schema should pass it.
- `get_model_provider_service` addresses a service by name, and
  `resolve_provider_service` falls back to it before concluding "not found" — a
  named service should never be judged absent on the strength of a listing that
  might be incomplete.
- The entry-parsing moves to `_provider_service_entry`, shared by both paths.

Verified against eng-ml-inference.staging: the listing goes from 16 to 30 services,
`main.tien_le.openai_all` appears, `parent="main.tien_le"` returns exactly the
three services in that schema, and resolving a genuinely absent name still errors.

Tests: +8. `test_follows_next_page_token` was checked against the old single-page
behavior, where it fails with "Right contains one more item: 'main.s.two'" — the
same page-2 loss that caused the report.

Co-authored-by: Isaac
@tt-le
tt-le force-pushed the tien/mps-listing-pagination branch from ab8c692 to 85b9768 Compare August 5, 2026 18:02
@andy-xu-db
andy-xu-db self-requested a review August 5, 2026 18:36
@tt-le
tt-le merged commit 7d0aca2 into main Aug 5, 2026
2 checks passed
@tt-le
tt-le deleted the tien/mps-listing-pagination branch August 5, 2026 19:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants