Skip to content

fix(tests): select a registry row explicitly when none auto-selects - #5914

Merged
mmabrouk merged 3 commits into
release/v0.112.0from
test-fix/useapi-selection-fallback
Aug 10, 2026
Merged

fix(tests): select a registry row explicitly when none auto-selects#5914
mmabrouk merged 3 commits into
release/v0.112.0from
test-fix/useapi-selection-fallback

Conversation

@mmabrouk

@mmabrouk mmabrouk commented Aug 10, 2026

Copy link
Copy Markdown
Member

Context

The readiness wait added in #5905's review round (wait for a checked row before opening the Use API drawer) assumed the registry auto-selects a row. CI's fresh project never checks one, so both use-api tests fail on the wait itself (gate run). My regression, honestly labeled.

Changes

Keep the race protection (the drawer captures the revision at click time), but when no row is checked, click the first row's checkbox to create the selection, then require it. Works in both auto-selecting and non-selecting environments.

Tests

Prettier clean. CI on this PR validates both use-api tests in the exact environment that failed.

Update (release-gate decision)

Per the release-gate call, this PR now also skips five tests with tracking comments, none deleted:

  • Rotating environment-sensitive failures from the final gate run: the skill-package upload test, the model-hub custom-provider test, and the observability trace-after-run test.
  • Eternity-class runtimes that dominate the CI wall clock: observability "view traces" and auto-evaluation "should run a single evaluation" (7-minute budgets, up to ~21 minutes with retries).

The gate's meaning after this PR: the acceptance suite's passing set is deterministic; the skipped five are tracked repair backlog, visible as skips rather than masquerading as gate noise.

The readiness wait added for the drawer-race review finding assumed the
registry auto-selects a row; CI's fresh project never checks a checkbox,
so the wait timed out (gate run 31401605372). Keep the race protection
but create the selection when it is absent.
@vercel

vercel Bot commented Aug 10, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agenta-documentation Error Error Aug 10, 2026 4:25pm

Request Review

@dosubot dosubot Bot added size:XS This PR changes 0-9 lines, ignoring generated files. tests labels Aug 10, 2026
@coderabbitai

coderabbitai Bot commented Aug 10, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: ce48afad-3d66-40f3-a9e4-9ae292b13783

📥 Commits

Reviewing files that changed from the base of the PR and between a8fe22a and 40adf8b.

📒 Files selected for processing (1)
  • web/oss/tests/playwright/acceptance/use-api/index.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • web/oss/tests/playwright/acceptance/use-api/index.ts

📝 Walkthrough

Summary by CodeRabbit

  • Bug Fixes

    • Improved API drawer interactions by ensuring the first variant is selected before the drawer opens.
    • Added checks to confirm the selection is active, making the workflow more reliable.
  • Tests

    • Temporarily skipped several unstable, slow, or environment-sensitive acceptance tests to improve release validation reliability.
    • Preserved the skipped test coverage for future re-enablement.

Walkthrough

The Playwright helper now explicitly selects the first variant row before opening the API drawer. Five acceptance tests are now skipped with comments that document runtime or environment-sensitive CI issues.

Changes

Playwright acceptance test stabilization

Layer / File(s) Summary
Use API drawer selection
web/oss/tests/playwright/acceptance/use-api/index.ts
The helper checks table-body row selection, selects the first variant checkbox when needed, and waits for the checked state.
Acceptance test quarantine
web/oss/tests/playwright/acceptance/agent-skills/skill-folder-upload.spec.ts, web/oss/tests/playwright/acceptance/auto-evaluation/index.ts, web/oss/tests/playwright/acceptance/observability/index.ts, web/oss/tests/playwright/acceptance/settings/model-hub.ts
Five tests now use skip registration. Comments document runtime or environment-sensitive CI issues and tracking status.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related issues

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly describes the main change: explicitly selecting a registry row when automatic selection does not occur.
Description check ✅ Passed The description directly explains the registry-selection fix, test skips, release-gate decision, and validation performed.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch test-fix/useapi-selection-fallback

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

…se-gate decision)

Per Mahmoud, 2026-08-10: the acceptance gate's remaining red is a rotating
tail of environment-sensitive tests (a different set each run) plus two
tests whose 7-minute budgets and retries dominate the wall clock. Skip
them with tracking comments rather than let them define the gate; each
stays in the file for repair.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: 06d547a0-e0ea-4fdd-bf4c-6f0aae39ad17

📥 Commits

Reviewing files that changed from the base of the PR and between 965851e and 7611963.

📒 Files selected for processing (1)
  • web/oss/tests/playwright/acceptance/use-api/index.ts

Comment thread web/oss/tests/playwright/acceptance/use-api/index.ts Outdated
@github-actions

github-actions Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Railway Preview Environment

Image tag pr-5914-0a0d568
Status Failed
Logs View workflow run
Updated at 2026-08-10T16:36:18.025Z

@mmabrouk
mmabrouk merged commit 7743514 into release/v0.112.0 Aug 10, 2026
41 of 43 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XS This PR changes 0-9 lines, ignoring generated files. tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant