[5650] chore(ci): make clone mode the only Railway preview path (WP4) - #5678
Conversation
dosubot
Bot
added
the
size:XL
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughSummary by CodeRabbit
WalkthroughThis change completes the Railway preview cutover to clone-only environments. Workflows require pinned image tags, use configurable template locations, verify cloned environments, and clean up cloned environments. The template supports separate wrapper-image tags, and documentation records the new lifecycle. ChangesRailway preview cutover
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant PRWorkflow
participant RailwaySetup
participant CloneScript
participant RailwayDeploy
participant RailwayCleanup
PRWorkflow->>RailwaySetup: pass build outputs and pinned image tag
RailwaySetup->>CloneScript: create preview clone and apply CI auth key
CloneScript->>RailwayDeploy: provide cloned environment
RailwayDeploy->>PRWorkflow: return preview URL and verification logs
PRWorkflow->>RailwayCleanup: request cleanup
RailwayCleanup->>CloneScript: destroy clone and sweep stale environments
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Railway Preview Environment
Updated at 2026-08-03T16:06:37.114Z |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 3b4f322c-925c-4e30-972a-dff9dd858557
📒 Files selected for processing (15)
.github/workflows/14-check-pr-preview.yml.github/workflows/41-railway-setup.yml.github/workflows/43-railway-deploy.yml.github/workflows/45-railway-cleanup.ymldocs/design/railway-preview-clone-spike/HANDOFF.mdhosting/railway/oss/README.mdhosting/railway/oss/scripts/preview-cleanup-stale.shhosting/railway/oss/scripts/preview-clone-create.shhosting/railway/oss/scripts/preview-clone-destroy.shhosting/railway/oss/scripts/preview-create-or-update.shhosting/railway/oss/scripts/preview-destroy.shhosting/railway/oss/scripts/preview-resolve-env.shhosting/railway/oss/template/README.mdhosting/railway/oss/template/apply.shhosting/railway/oss/template/template.json
💤 Files with no reviewable changes (4)
- hosting/railway/oss/scripts/preview-destroy.sh
- hosting/railway/oss/scripts/preview-create-or-update.sh
- hosting/railway/oss/scripts/preview-cleanup-stale.sh
- hosting/railway/oss/scripts/preview-resolve-env.sh
|
First soak finding, fixed in-place: the first clone-mode preview ran green through clone/deploy/smoke, but the acceptance suites failed with 401s — legacy previews wrote the CI test auth key into every environment at configure time, while clones inherit the template's own generated key. Fix (commit c827192): workflow 41 passes |
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: dd54eef3-1602-465b-a4f1-27e6d6bf8c74
📒 Files selected for processing (2)
.github/workflows/41-railway-setup.ymlhosting/railway/oss/scripts/preview-clone-create.sh
🚧 Files skipped from review as they are similar to previous changes (1)
- hosting/railway/oss/scripts/preview-clone-create.sh
|
Second and third soak findings, both fixed. The first clone-mode previews surfaced two gaps where the template did not carry configuration that legacy
Both were pre-existing gaps between the template and legacy behavior, not defects in the clone mechanism itself: cloning, patching, deploying, smoke-testing, and teardown have worked in every run. This is exactly what the soak was for. |
|
Five flaky web acceptance tests disabled at Mahmoud's direction, tracked in #5695 with the evidence and the re-enable criteria: app deletion, chat-prompt app creation, human-annotation mismatched testset, and two playground run-variant tests. Each is skipped at the declaration with a one-line reason pointing at the issue. They failed the same way under the old preview system, so this is not a regression from the cutover. Also in this push: the runner healthcheck is removed from the template, since the runner listens on 8765 with no |
|
CodeRabbit findings addressed (commit a0199ce):
Worth flagging that the first one mattered: the auth-key fix earlier in this PR is what took the API suite from ten failures to green, and this missing declaration would have quietly undone it on any future run. |
2 participants
Final step of the clone-based Railway preview redesign (#5650, WP4). Merge when the clone-mode soak looks good; rollback before merge is flipping
RAILWAY_PREVIEW_MODEback tolegacy, and rollback after merge is reverting this PR.Clone mode is already live via the repo variable. This PR removes the legacy preview path so clone is the only one, and finishes the cutover items the earlier PRs marked
TODO(cutover).What changes
modejob, the mode inputs, the Railway CLI installs, and every legacy setup/deploy/destroy step are gone. The stale-clone sweep and its daily cron remain. Job outputs that the tests job and the PR comment consume now come from the clone/verify steps directly.vars.RAILWAY_TEMPLATE_PROJECTandvars.RAILWAY_TEMPLATE_ENV(defaults: the current template project), so moving the template later is a variable change plus oneprojectline intemplate.json, no code edits.template.jsonmoves tov0.108.0app images and per-service content-addressed wrapper tags (the images workflow 42 publishes);apply.shresolves per-service wrapper tags from the definition. The live template already matches this definition (apply.sh --dry-runis CLEAN), so the daily drift check goes green with this merge.preview-create-or-update.sh,preview-resolve-env.sh,preview-destroy.sh,preview-cleanup-stale.sh), each with a repo-wide grep proving zero remaining references.What deliberately stays
bootstrap.sh,configure.sh,deploy-from-images.sh,upgrade.sh,smoke.sh, and the wrapper image sources stay untouched: they serve self-hosters deploying Agenta on their own Railway, the template-maintenance path, and workflow 42's byte-faithfulness guard readsdeploy-from-images.shdirectly.Verification
bash -nand shellcheck clean on all touched scripts; actionlint clean on all four workflows;template.jsonparses and substitutes with zero leftover placeholders.Closes #5650
https://claude.ai/code/session_011zdniFW44QDqcs4Nz8n3hr