Skip to content

docs(flaky-tests): resync Test Collections and the migration guide - #362

Merged
TylerJang27 merged 4 commits into
mainfrom
docs/test-collections-sept-drift
Sep 21, 2026
Merged

TylerJang27 merged 4 commits into
mainfrom
docs/test-collections-sept-drift

Conversation

@TylerJang27

Copy link
Copy Markdown
Contributor

A week of trunk2 changes (2026-09-14 → 09-21) left the Test Collections instructions and the migration guide stating things the product no longer does. This corrects them, and refreshes the published v2 OpenAPI spec.

Important

Blocked on trunk-io/trunk2#6197. The quarantining copy here describes the behaviour that PR introduces — seeded on only for a fully-migrated org. Merging this first would put the opposite claim in front of every migrating customer. Everything else in the diff is independent; say the word and I'll split the quarantining copy into a follow-up so the rest can ship now.

What was wrong

Was Now
Setup checklist "It tracks four steps" Five — TICKETING_REVIEWED joined (#5907, #5965)
New collection "a basic set of flake-detection monitors" The five seeded monitors, with their configs and branch patterns (#5959)
Tabs after creation "Tests and Uploads stay disabled until test results arrive" Uploads opens on the upload, Tests on ingestion (#5962)
"Nothing carries over from your repositories" Nothing did Monitors (#6068) and ticketing connections (#5905, #5906) both can, deliberately
Permissions table No ticketing or monitor rows Both, all member-level (#6013)
Detection Default state Repository values, read as universal A new collection's

Navigation that no longer resolves

Seven pages gave Settings → Repositories → [repository] → … as the only route to a setting. For an org that has fully migrated, the repository name on that page is plain text (#5961), so the route has no first click. The quarantining page, the audit-log section and the three ticketing integration pages now lead with the collection or organization path and keep the repository one as a parenthetical.

The three ticketing pages were the sharpest: none mentioned the organization path at all, while automatic-ticketing.mdx — the page that documents it — sends readers to them for setup.

v2 OpenAPI spec

Straight re-copy of the generated ts/apps/public-api/openapi-v2.json, last refreshed 2026-09-16. Adds testCases.linkTicket (#6020) and pullRequests.queueEntryHistoryBatch (#6124), 21 missing schemas and 10 changed ones. linkTicket is added to the nav in docs.json.

Screenshot

One new image, on the migrate-monitors section. Reviewing the first pair turned up a group header count that did not match its rows; that is fixed in the product and these are the corrected shots.

Getting Started

Step 2 never told a new organization to create a collection or get its ID, then Steps 3 and 4 both sent it to that collection's tabs. Step 4 also still said pass-on-retry was the only thing enabled by default.

Deliberately unchanged

  • flaky-tests/webhooks/index.mdx, automatic-ticketing.mdx, dashboard.mdx, the CLI reference, and the collection section of quarantining/index.mdx were all checked against main and are accurate.
  • get-started/multiple-repositories.mdx still documents the per-repository fork-upload identifier. trunk-io/trunk2#6188–6190 replace it with a per-collection opt-in keyed on the collection's own shortId, but they are drafts — this needs a rewrite when they land, not before.

🤖 Generated with Claude Code

TylerJang27 and others added 4 commits September 21, 2026 15:28
Collections now seed five monitors and quarantining, the setup checklist
has five steps, monitors and ticketing connections can be brought across
from a repository, and the repo-scoped settings routes close once an org
is fully migrated. Refreshes openapi-v2.json, which was missing
testCases.linkTicket and pullRequests.queueEntryHistoryBatch.

Screenshots are outstanding — see TODO-SCREENSHOTS.md.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Corrects the quarantining default (seeded since 2026-09-16, no backfill
for older collections, no org-status gate), makes the detection default
state column collection-first, streamlines test-collections.mdx, reverts
the alt text and the fully-migrated paragraph, and drops the screenshot
TODO file.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Embeds the migrate-monitors screenshots, drops the repository note from
the detection default-state column, and aligns the quarantining copy with
trunk-io/trunk2#6197, which seeds quarantining on only for a fully
migrated org. A migrating org's new collections start with it off, so the
migration guide's quarantining step returns to enabling it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The earlier pair showed a group header count that did not match its rows.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@mintlify

mintlify Bot commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
trunk 🟢 Ready View Preview Sep 21, 2026, 4:37 PM

@TylerJang27
TylerJang27 merged commit deb79d8 into main Sep 21, 2026
3 checks passed
@mintlify

mintlify Bot commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

Docs PR opened: #363

Added a weekly changelog entry summarizing Test Collections migration tools, onboarding polish, permissions changes, and two new v2 API endpoints.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant