[fix] Round 2: fix agent-chat playground navigation, app-overview app pollution, use-api selectors - #5905
Merged
Merged
Conversation
Same two issues fixed elsewhere in the acceptance suite (#5900), now hit by use-api's Use API drawer tests too: - .ant-drawer-content-wrapper never matches the drawer, which renders through EnhancedDrawer (a facade over the @agenta/ui Radix Sheet, [data-slot="sheet-content"]). - waitForLoadState("networkidle") hangs forever because the layout keeps a live SSE connection open (ProjectWatch). Verified live: both use-api tests passed (13.7s, 9.2s).
Root cause of the still-failing overview test after #5900's networkidle fix: apiHelpers.getApp() picks the first workflow matching a loose "completion" filter (not is_chat/is_custom/is_evaluator), but the workflow list API returns no field distinguishing a completion app from an agent app — both carry {is_application, is_evaluator, is_snippet}. In CI's single shared ephemeral project, agent-chat's test runs first and leaves behind an agent app that getApp() can pick up here instead of creating a real completion app. An agent app has no Deployment section (AgentOverview renders instead), so the test hung 3x120s waiting for a heading that would never appear. createApp("completion") always mints a fresh app, sidestepping the classification ambiguity entirely. Verified live: reproduced the exact CI scenario (seeding an agent app via agent-chat's fixture in the same ephemeral project, then running this test right after) and confirmed it now passes.
… entity-resolution races
Two bugs in navigateToAgentPlayground, both diagnosed live against the
dev stack:
1. URL construction bug (the actual CI failure, gate job 31390095129).
scopedPrefix was read from page.url() before the first navigation —
on a fresh test page.url() is still "about:blank" (seedAgentChatApp
only makes API calls), so scopedPrefix was permanently "". Every
later navigation, including the playground goto, built an unscoped
URL like /apps/<id>/playground instead of
/w/<ws>/p/<proj>/apps/<id>/playground. The app doesn't route that to
the seeded app; it falls through to the bare project-level
onboarding playground ("What do you want to build?"), which has no
composer textbox, hanging expectPath for the full timeout — exactly
matching the gate log's redirect chain
(.../w?revisions=... -> .../apps -> bare .../playground).
Fix: read scopedPrefix from page.url() again AFTER the first
navigation resolves it.
2. Read-your-writes race. A revision committed moments earlier via
direct API calls can race the playground's entity resolution,
landing on MainLayout's "Playground is unable to communicate with
the service" error state instead of the composer. That panel's own
"Try again" button has no onClick handler (dead), so a real reload
is used to retry instead.
Verified live against the dev stack: reproduced the exact CI redirect
chain with the bug in place, confirmed the fix resolves it, then hit
the race (3/3 fresh-context runs) and confirmed the reload recovery
resolves it too (3/3 clean runs afterward, ~10s each).
FileCard (@ant-design/x) renders an image attachment as a bare <img> with the filename only in alt text — there is no separate visible text caption for images (only file/audio/video card types render a name label). The post-send and post-reload assertions used getByText, which never matches; switched to getByAltText, matching the pattern the pre-send assertion on the same page already used correctly. Also bumps the test timeout to 120s: API seeding + 2 navigations + file upload + SSE-mocked run + reload is heavier than the 60s default, and siblings doing less than this already use 120s+ (#5695). Verified live: full test (upload, send, verify render, reload, verify persists) passed 3/3 runs against the dev stack.
dosubot
Bot
added
the
size:XS
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository YAML (base), Organization UI (inherited) Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughSummary by CodeRabbit
WalkthroughPlaywright acceptance tests now resolve scoped agent playground URLs, retry transient playground errors, use stable attachment and drawer selectors, create compatible completion apps, and adjust test timeouts. ChangesPlaywright acceptance flows
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related issues
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Organization UI (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: f1543bd9-c129-454b-8ee7-cd99cbf7d9f9
📒 Files selected for processing (4)
web/oss/tests/playwright/acceptance/agent-chat/attach-send-render-reload.spec.tsweb/oss/tests/playwright/acceptance/agent-chat/tests.tsweb/oss/tests/playwright/acceptance/app/app-management.tsweb/oss/tests/playwright/acceptance/use-api/index.ts
Contributor
Railway Preview Environment
Updated at 2026-08-10T15:04:25.097Z |
The button is not disabled while no revision is selected and captures the revision at click time; clicking in the pre-selection window opens the drawer with an undefined revision and empty snippets. Wait for the antd row checkbox to turn checked first.
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
Round 2 of the 0.112.0 acceptance gate cleanup, following #5900/#5901. The gate re-ran with those fixes: 43 passed, 4 flaky-recovered, 4 failed. Two of the four were the follow-up already flagged in #5900 (
use-api's same stale selector / hanging wait). The other two needed fresh diagnosis: the app-overview test still failed after its round-1 fix, and agent-chat was still unreproducible locally until this round found the actual bug. All four are fixed here and verified live against the dev stack (144.76.237.122:8180).Changes
use-apidrawer tests. Same two issues as #5900, on a file that wasn't in scope for round 1:.ant-drawer-content-wrappernever matches (the drawer is[data-slot="sheet-content"], an@agenta/uiSheet), andwaitForLoadState("networkidle")hangs forever because the layout keeps a live SSE connection open (ProjectWatch). Both fixed the same way as the round-1 overview test.App overview test — the real root cause. #5900 removed the hanging
networkidlewait, but CI kept failing at the same spot ("waiting for getByRole('heading', {name: 'Deployment'})", 120s × 3). The test calledapiHelpers.getApp(), whose "completion" matcher only excludesis_chat/is_custom/is_evaluator— but the/workflows/querylist API returns no field that distinguishes a completion app from an agent app; both carry identical{is_application, is_evaluator, is_snippet}flags. In CI's single shared ephemeral project (one project for all 80 tests),agent-chat's test runs immediately before this one and leaves behind an agent app.getApp()picked that up instead of creating a real completion app. An agent app rendersAgentOverview, notDeploymentOverview— no "Deployment" heading ever appears, hence the deterministic 120s hang.Fix:
apiHelpers.createApp("completion")instead ofgetApp()— always mints a fresh app, sidestepping the classification ambiguity. Verified by reproducing the exact CI scenario: seed an agent app viaagent-chat's fixture in one ephemeral project, then run this test right after in the same project. Failed before the fix, passed after.Agent-chat: the actual bug behind "unreproducible locally." Round 1 replayed the seed+navigate flow live and it worked, so I couldn't reproduce the CI failure and left the test alone (documented in #5900). This round's gate log gave the missing piece: the CI navigation chain shows
/w?revisions=...→ workspace →/apps→ a bare/playground(missing/apps/<id>). Root cause:seedAgentChatApp()only makes API calls (no navigation), so on a fresh testpage.url()is still"about:blank"when this line runs —scopedPrefixis permanently"". Both the firstgoto("/apps")and, critically, the secondgoto(to the actual playground) use this same stale empty prefix, so the playground navigation targets an unscoped/apps/<id>/playground— a URL the app doesn't route to the seeded app; it falls through to the bare onboarding playground ("What do you want to build?"), which has no composer, hangingexpectPathfor the full timeout.Fix: recompute
scopedPrefixfrompage.url()again after the first navigation resolves it to the real workspace/project-scoped URL.That got the test past navigation, but exposed a second, real issue: a revision committed moments earlier via direct API calls can race the playground's entity resolution, landing on
MainLayout's "Playground is unable to communicate with the service" error state instead of the composer (3/3 reproduction rate in fresh browser contexts). That panel's own "Try again" button has noonClickhandler (web/oss/src/components/Playground/Components/MainLayout/index.tsx:318—<Button>Try again</Button>, literally does nothing), so I added a realpage.reload()retry in the test instead, which re-runs entity resolution from scratch. 3/3 clean after that.Finally, the test's own assertions were stale:
FileCard(@ant-design/x) renders an image attachment as a bare<img>with the filename only inalt, not as visible text (only file/audio/video card types get a name caption).getByText(IMAGE_NAME)never matched post-send; switched togetByAltText, matching the pre-send assertion on the same page that already used it correctly. Also bumped the timeout to 120s (siblings doing less already use 120s+, per #5695) since this flow is API seed + 2 navigations + upload + SSE-mocked run + reload.Live verification summary
use-api: variant TypeScript snippetuse-api: deployment TypeScript snippetNotes for reviewers
MainLayout/index.tsx:318) is a separate, minor product bug (button exists but does nothing) — not fixed here since I have no way to verify a product-code change against this dev stack without deploying to it, which I was asked not to do. Flagging for a follow-up.144.76.237.122:8180throughout; made no product-code changes, no stack deploys/restarts. One accidental cross-navigation into another agent's live session during manual diagnosis (read-only, no clicks) — caught and backed out immediately.