docs: add unlisted Shorebird Zap prototype page - #676
Conversation
Adds documentation for Shorebird Zap, the prototype for previewing and iterating on Flutter projects from a mobile device. The page is intentionally unlisted: hidden from the sidebar, excluded from Pagefind search, and served with noindex/nofollow, since Zap is only available to a limited set of customers. Also excludes the page from the generated llms-small.txt output. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Nice page. The structure and the core-cycle framing read well, and the prototype framing is right. Two things need work before this ships: "unlisted" isn't actually achieved (the full page leaks through three separate channels), and CI is red on three checks.
I built the branch and verified every finding and every fix below.
The three leaks
The page is correctly hidden from the sidebar and from Pagefind. But it's still published in full in three places.
1. llms-full.txt still contains the whole page
dist/llms-full.txt line 11307 has # Shorebird Zap and the complete body. This isn't something exclude can fix. It's deliberate upstream. From starlight-llms-txt's 0.2.1 changelog:
Fixes a bug where pages excluded using the
excludeconfiguration option were excluded inllms-full.txtinstead of only inllms-small.txt
So exclude means "exclude from llms-small.txt", and always will. (draft: true filters both, but Starlight drops draft pages from production builds entirely, so /zap/ would 404, which is not an option.)
Fix: keep exclude for llms-small.txt and strip llms-full.txt after the build, giving the plugin an unambiguous separator so the strip can't cut in the wrong place. In astro.config.mjs:
import { readFile, writeFile } from 'node:fs/promises';
import { globSync } from 'node:fs';
import { unlistedPages } from './src/unlisted.ts';
// `starlight-llms-txt` joins pages with this string. The default is a bare
// blank line, indistinguishable from a paragraph break; an HTML comment gives
// the strip below a reliable boundary. Markdown renderers ignore it.
const llmsPageSeparator = '\n\n<!-- page -->\n\n';
// The plugin's `exclude` option only filters `llms-small.txt` (deliberate
// upstream, see its 0.2.1 changelog), so `llms-full.txt` is filtered here.
const stripUnlistedFromLlmsFull = {
name: 'strip-unlisted-from-llms-full',
hooks: {
'astro:build:done': async ({ dir, logger }) => {
const file = new URL('llms-full.txt', dir);
const pages = (await readFile(file, 'utf8')).split(llmsPageSeparator);
// This hook runs after the content collection APIs are torn down, so
// titles are read straight off disk.
const titles = unlistedPages.map((id) => {
const [path] = globSync(`src/content/docs/${id}.{md,mdx}`);
if (!path) throw new Error(`Unlisted page "${id}" not found`);
const match = /^title:\s*(.+)$/m.exec(readFileSync(path, 'utf8'));
if (!match) throw new Error(`No title in ${path}`);
return match[1].trim().replace(/^['"]|['"]$/g, '');
});
const kept = pages.filter(
(page) => !titles.some((title) => page.startsWith(`# ${title}\n`)),
);
const removed = pages.length - kept.length;
if (removed !== titles.length) {
throw new Error(
`llms-full.txt: expected to strip ${titles.length} unlisted page(s), stripped ${removed}.`,
);
}
await writeFile(file, kept.join(llmsPageSeparator));
logger.info(`Stripped ${removed} unlisted page(s) from llms-full.txt`);
},
},
};Add stripUnlistedFromLlmsFull to integrations. It throws rather than silently passing if the plugin's output format changes, so it can't rot quietly.
Since three places need the same list, put it in one file, the new src/unlisted.ts:
// Pages that are reachable only by their direct URL. They are kept out of the
// sidebar, Pagefind, the sitemap, the `llms*.txt` bundles, and the
// agent-facing `.md` routes, so nothing advertises them.
//
// Values are content collection entry IDs, i.e. the path under
// `src/content/docs/` without its extension (`zap.mdx` -> `zap`).
export const unlistedPages = ['zap'];2. The full page is published as Markdown at /zap.md
dist/zap.md exists on this branch. src/pages/[...slug].md.ts:23 filters only on draft, so the agent-facing Markdown route hands out the complete page text. This is the leak most likely to actually be hit, since it's what agents fetch.
Fix in src/pages/[...slug].md.ts:
import { unlistedPages } from '../unlisted';
// Unlisted pages are reachable at their HTML URL but are not published as
// agent-facing Markdown, which would otherwise hand out the full text of a
// page that is deliberately not advertised.
const docs = await getCollection(
'docs',
(entry) => !entry.data.draft && !unlistedPages.includes(entry.id),
);3. The sitemap advertises the page to crawlers
dist/sitemap-0.xml contains https://docs.shorebird.dev/zap/. The noindex, nofollow meta does render correctly, but the sitemap actively points crawlers at a page we're telling them to ignore.
Fix: Starlight only adds @astrojs/sitemap if it isn't already in integrations (@astrojs/starlight/dist/index.js:55), so adding it ourselves with a filter is the supported way in:
import sitemap from '@astrojs/sitemap';
// Starlight adds `@astrojs/sitemap` itself unless it is already in this array,
// so configuring it here is what lets unlisted pages be filtered out.
sitemap({
filter: (page) =>
!unlistedPages.some(
(id) => page === `${site}${id}/` || page === `${site}${id}`,
),
}),One catch: @astrojs/sitemap is currently only a transitive dependency, so also run npm i @astrojs/sitemap to make it direct.
CI is red on three checks
The description says npm run build passes, but CI runs format:check before astro build, so the build step never ran. See run 35282546988.
Prettier: fix with npm run format. Purely mechanical (unwrapped prose, trailing whitespace, a long import).
Vale, 11 errors: two different causes, two different fixes.
Sentence-case headings (lines 29, 83, 86, 128): these fail because Zap is a product name Vale doesn't know. Don't lowercase it; teach Vale instead. One line in .vale/styles/Shorebird/Headings.yml:
exceptions:
- Shorebird
- Shorebird's
- ZapFirst person (lines 54, 88, 154, 167, 170, 179 x2): house style is second person, and existing docs phrase this as "Reach out on Discord" with no "us". Concrete rewrites:
| Line | Now | Change to |
|---|---|---|
| 54 | "We'll support other coding agents and platforms (Cursor, Codex, Antigravity) soon." | "Support for other coding agents and platforms (Cursor, Codex, Antigravity) is coming soon." |
| 88 | "In the future, we plan to support OpenAI's Codex, ..." | "Support for OpenAI's Codex, Cursor, Google's Antigravity, and any other cloud-based coding agent is planned." |
| 154 | "one that worked for us is" | "one that has worked well is" |
| 167 | "please let us know via the Feedback form" | "please reach out via the Feedback form" |
| 170 | "My agent can't access the link" | "The agent can't access the link" |
| 179 | "Reach out to us on Discord or fill out our Feedback form" | "Reach out on Discord or fill out the Feedback form" |
cspell: Github -> GitHub at lines 103 and 117. (CI's spell-check passed, but npm run lint:content catches it locally.)
Verified working, no action needed
Internal links all valid, the #skip-mcp anchor resolves, Pagefind exclusion works, and every external link resolves except the Play Store listing (404 anonymously, which is expected for a closed track, and the troubleshooting entry already covers it). mcp.shorebird.dev/mcp returning 405 to a GET is normal for an MCP endpoint.
After all the fixes above: build passes, Vale/prettier/cspell/component-labels all clean, /zap/ still builds, and Zap appears in zero of llms-full.txt, llms-small.txt, sitemap-0.xml, /zap.md, and the search index.
Happy to push a branch with all of this applied if that's easier than working through it. Just say the word.
Hiding the page from the sidebar and Pagefind still left it published in
full through three other channels. A new `src/unlisted.ts` list now drives
all of them:
- llms-full.txt: the plugin's `exclude` only filters llms-small.txt, so
a build hook strips unlisted pages from llms-full.txt. They are found by
position (`demote` sorts them last), not by title, since titles such as
"Overview" are shared by several pages. The hook fails the build if the
page it would strip is not the expected one.
- /zap.md: the agent-facing Markdown route now skips unlisted pages, and
Head.astro no longer emits a `rel="alternate"` link pointing at it.
- sitemap: `@astrojs/sitemap` is configured explicitly (now a direct
dependency) with a filter, which stops Starlight adding its own.
Page fixes: prettier formatting, second-person phrasing and sentence-case
headings for Vale ("Zap" added to the heading exceptions), GitHub casing,
h1 headings demoted below the page title, the visible iOS TODO removed,
and the screenshots moved to src/assets with descriptive names.
Checked against the Zap app source: the app is named "Shorebird Zap", the floating button reads "Field note"/"Note" rather than "Notes", and its panel says "Back to projects". The session button is "Start Session", the markup button is "Mark up", sign-in is "Sign in", and the escape gesture is a three-finger hold of about a second.
This PR adds documentation for Shorebird Zap, our prototype for previewing and iterating on Flutter projects from a mobile device.
The page is intentionally unlisted since Zap is only an early preview. It's reachable only at its direct URL,
/zap/, and nothing on the site advertises it.Changes
src/content/docs/zap.mdx: new page, hidden from the sidebar and Pagefind, withnoindex, nofollow. UI wording checked against the Zap app source.src/assets/zap_add_custom_connector.png,zap_custom_connector_form.png: MCP connector setup screenshotssrc/unlisted.ts: the one list of unlisted pages, used by everything belowastro.config.mjs:@astrojs/sitemapconfigured explicitly with a filter, so unlisted pages are left out of the sitemap (Starlight only adds its own copy when this one is absent). It is now a direct dependency inpackage.json.starlight-llms-txt'sexcludekeeps unlisted pages out ofllms-small.txt.excludedeliberately does not apply tollms-full.txt(see the plugin's 0.2.1 changelog), so a build hook strips them from that file. It finds them by position (demotesorts them last), not by title, because titles repeat. It fails the build if the page it would strip is not the expected one.src/pages/[...slug].md.ts: no agent-facing/zap.mdsrc/components/starlight/Head.astro: norel="alternate"Markdown link on unlisted pages, since that file no longer exists.vale/styles/Shorebird/Headings.yml:Zapadded as a proper nounVerification
npm run buildpasses, all internal links are valid, and prettier, Vale, cspell and the component-label lint are clean. In the built site,/zap/exists, and Zap appears in none ofllms.txt,llms-small.txt,llms-full.txt,sitemap-0.xml, the Pagefind index, or any.mdfile. Every other page is still inllms-full.txt, in the same order as production.Notes for reviewers
The iOS install tab only has the TestFlight step for now. The remaining steps get added once the app is listed and they can be verified firsthand.
🤖 Generated with Claude Code, heavily edited by Wesley Peck because robots have no soul