Skip to content

docs: add unlisted Shorebird Zap prototype page - #676

Merged
AbhishekDoshi26 merged 4 commits into
mainfrom
docs/zap
Sep 19, 2026
Merged

AbhishekDoshi26 merged 4 commits into
mainfrom
docs/zap

Conversation

@wrpeck

@wrpeck wrpeck commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

This PR adds documentation for Shorebird Zap, our prototype for previewing and iterating on Flutter projects from a mobile device.

The page is intentionally unlisted since Zap is only an early preview. It's reachable only at its direct URL, /zap/, and nothing on the site advertises it.

Changes

  • src/content/docs/zap.mdx: new page, hidden from the sidebar and Pagefind, with noindex, nofollow. UI wording checked against the Zap app source.
  • src/assets/zap_add_custom_connector.png, zap_custom_connector_form.png: MCP connector setup screenshots
  • src/unlisted.ts: the one list of unlisted pages, used by everything below
  • astro.config.mjs:
    • @astrojs/sitemap configured explicitly with a filter, so unlisted pages are left out of the sitemap (Starlight only adds its own copy when this one is absent). It is now a direct dependency in package.json.
    • starlight-llms-txt's exclude keeps unlisted pages out of llms-small.txt. exclude deliberately does not apply to llms-full.txt (see the plugin's 0.2.1 changelog), so a build hook strips them from that file. It finds them by position (demote sorts them last), not by title, because titles repeat. It fails the build if the page it would strip is not the expected one.
  • src/pages/[...slug].md.ts: no agent-facing /zap.md
  • src/components/starlight/Head.astro: no rel="alternate" Markdown link on unlisted pages, since that file no longer exists
  • .vale/styles/Shorebird/Headings.yml: Zap added as a proper noun

Verification

npm run build passes, all internal links are valid, and prettier, Vale, cspell and the component-label lint are clean. In the built site, /zap/ exists, and Zap appears in none of llms.txt, llms-small.txt, llms-full.txt, sitemap-0.xml, the Pagefind index, or any .md file. Every other page is still in llms-full.txt, in the same order as production.

Notes for reviewers

The iOS install tab only has the TestFlight step for now. The remaining steps get added once the app is listed and they can be verified firsthand.

🤖 Generated with Claude Code, heavily edited by Wesley Peck because robots have no soul

Adds documentation for Shorebird Zap, the prototype for previewing and
iterating on Flutter projects from a mobile device.

The page is intentionally unlisted: hidden from the sidebar, excluded
from Pagefind search, and served with noindex/nofollow, since Zap is
only available to a limited set of customers.

Also excludes the page from the generated llms-small.txt output.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@AbhishekDoshi26 AbhishekDoshi26 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice page. The structure and the core-cycle framing read well, and the prototype framing is right. Two things need work before this ships: "unlisted" isn't actually achieved (the full page leaks through three separate channels), and CI is red on three checks.

I built the branch and verified every finding and every fix below.


The three leaks

The page is correctly hidden from the sidebar and from Pagefind. But it's still published in full in three places.

1. llms-full.txt still contains the whole page

dist/llms-full.txt line 11307 has # Shorebird Zap and the complete body. This isn't something exclude can fix. It's deliberate upstream. From starlight-llms-txt's 0.2.1 changelog:

Fixes a bug where pages excluded using the exclude configuration option were excluded in llms-full.txt instead of only in llms-small.txt

So exclude means "exclude from llms-small.txt", and always will. (draft: true filters both, but Starlight drops draft pages from production builds entirely, so /zap/ would 404, which is not an option.)

Fix: keep exclude for llms-small.txt and strip llms-full.txt after the build, giving the plugin an unambiguous separator so the strip can't cut in the wrong place. In astro.config.mjs:

import { readFile, writeFile } from 'node:fs/promises';
import { globSync } from 'node:fs';
import { unlistedPages } from './src/unlisted.ts';

// `starlight-llms-txt` joins pages with this string. The default is a bare
// blank line, indistinguishable from a paragraph break; an HTML comment gives
// the strip below a reliable boundary. Markdown renderers ignore it.
const llmsPageSeparator = '\n\n<!-- page -->\n\n';

// The plugin's `exclude` option only filters `llms-small.txt` (deliberate
// upstream, see its 0.2.1 changelog), so `llms-full.txt` is filtered here.
const stripUnlistedFromLlmsFull = {
  name: 'strip-unlisted-from-llms-full',
  hooks: {
    'astro:build:done': async ({ dir, logger }) => {
      const file = new URL('llms-full.txt', dir);
      const pages = (await readFile(file, 'utf8')).split(llmsPageSeparator);
      // This hook runs after the content collection APIs are torn down, so
      // titles are read straight off disk.
      const titles = unlistedPages.map((id) => {
        const [path] = globSync(`src/content/docs/${id}.{md,mdx}`);
        if (!path) throw new Error(`Unlisted page "${id}" not found`);
        const match = /^title:\s*(.+)$/m.exec(readFileSync(path, 'utf8'));
        if (!match) throw new Error(`No title in ${path}`);
        return match[1].trim().replace(/^['"]|['"]$/g, '');
      });
      const kept = pages.filter(
        (page) => !titles.some((title) => page.startsWith(`# ${title}\n`)),
      );
      const removed = pages.length - kept.length;
      if (removed !== titles.length) {
        throw new Error(
          `llms-full.txt: expected to strip ${titles.length} unlisted page(s), stripped ${removed}.`,
        );
      }
      await writeFile(file, kept.join(llmsPageSeparator));
      logger.info(`Stripped ${removed} unlisted page(s) from llms-full.txt`);
    },
  },
};

Add stripUnlistedFromLlmsFull to integrations. It throws rather than silently passing if the plugin's output format changes, so it can't rot quietly.

Since three places need the same list, put it in one file, the new src/unlisted.ts:

// Pages that are reachable only by their direct URL. They are kept out of the
// sidebar, Pagefind, the sitemap, the `llms*.txt` bundles, and the
// agent-facing `.md` routes, so nothing advertises them.
//
// Values are content collection entry IDs, i.e. the path under
// `src/content/docs/` without its extension (`zap.mdx` -> `zap`).
export const unlistedPages = ['zap'];

2. The full page is published as Markdown at /zap.md

dist/zap.md exists on this branch. src/pages/[...slug].md.ts:23 filters only on draft, so the agent-facing Markdown route hands out the complete page text. This is the leak most likely to actually be hit, since it's what agents fetch.

Fix in src/pages/[...slug].md.ts:

import { unlistedPages } from '../unlisted';

// Unlisted pages are reachable at their HTML URL but are not published as
// agent-facing Markdown, which would otherwise hand out the full text of a
// page that is deliberately not advertised.
const docs = await getCollection(
  'docs',
  (entry) => !entry.data.draft && !unlistedPages.includes(entry.id),
);

3. The sitemap advertises the page to crawlers

dist/sitemap-0.xml contains https://docs.shorebird.dev/zap/. The noindex, nofollow meta does render correctly, but the sitemap actively points crawlers at a page we're telling them to ignore.

Fix: Starlight only adds @astrojs/sitemap if it isn't already in integrations (@astrojs/starlight/dist/index.js:55), so adding it ourselves with a filter is the supported way in:

import sitemap from '@astrojs/sitemap';

// Starlight adds `@astrojs/sitemap` itself unless it is already in this array,
// so configuring it here is what lets unlisted pages be filtered out.
sitemap({
  filter: (page) =>
    !unlistedPages.some(
      (id) => page === `${site}${id}/` || page === `${site}${id}`,
    ),
}),

One catch: @astrojs/sitemap is currently only a transitive dependency, so also run npm i @astrojs/sitemap to make it direct.


CI is red on three checks

The description says npm run build passes, but CI runs format:check before astro build, so the build step never ran. See run 35282546988.

Prettier: fix with npm run format. Purely mechanical (unwrapped prose, trailing whitespace, a long import).

Vale, 11 errors: two different causes, two different fixes.

Sentence-case headings (lines 29, 83, 86, 128): these fail because Zap is a product name Vale doesn't know. Don't lowercase it; teach Vale instead. One line in .vale/styles/Shorebird/Headings.yml:

exceptions:
  - Shorebird
  - Shorebird's
  - Zap

First person (lines 54, 88, 154, 167, 170, 179 x2): house style is second person, and existing docs phrase this as "Reach out on Discord" with no "us". Concrete rewrites:

Line Now Change to
54 "We'll support other coding agents and platforms (Cursor, Codex, Antigravity) soon." "Support for other coding agents and platforms (Cursor, Codex, Antigravity) is coming soon."
88 "In the future, we plan to support OpenAI's Codex, ..." "Support for OpenAI's Codex, Cursor, Google's Antigravity, and any other cloud-based coding agent is planned."
154 "one that worked for us is" "one that has worked well is"
167 "please let us know via the Feedback form" "please reach out via the Feedback form"
170 "My agent can't access the link" "The agent can't access the link"
179 "Reach out to us on Discord or fill out our Feedback form" "Reach out on Discord or fill out the Feedback form"

cspell: Github -> GitHub at lines 103 and 117. (CI's spell-check passed, but npm run lint:content catches it locally.)


Verified working, no action needed

Internal links all valid, the #skip-mcp anchor resolves, Pagefind exclusion works, and every external link resolves except the Play Store listing (404 anonymously, which is expected for a closed track, and the troubleshooting entry already covers it). mcp.shorebird.dev/mcp returning 405 to a GET is normal for an MCP endpoint.

After all the fixes above: build passes, Vale/prettier/cspell/component-labels all clean, /zap/ still builds, and Zap appears in zero of llms-full.txt, llms-small.txt, sitemap-0.xml, /zap.md, and the search index.

Happy to push a branch with all of this applied if that's easier than working through it. Just say the word.

Comment thread astro.config.mjs Outdated
Comment thread src/content/docs/zap.mdx Outdated
Comment thread src/content/docs/zap.mdx
Comment thread src/content/docs/zap.mdx Outdated
Comment thread src/content/docs/zap.mdx Outdated
Comment thread src/content/docs/zap.mdx Outdated
Comment thread src/content/docs/zap.mdx Outdated
Comment thread src/content/docs/zap.mdx Outdated
Comment thread src/content/docs/zap.mdx Outdated
Comment thread src/content/docs/zap.mdx Outdated
Hiding the page from the sidebar and Pagefind still left it published in
full through three other channels. A new `src/unlisted.ts` list now drives
all of them:

- llms-full.txt: the plugin's `exclude` only filters llms-small.txt, so
  a build hook strips unlisted pages from llms-full.txt. They are found by
  position (`demote` sorts them last), not by title, since titles such as
  "Overview" are shared by several pages. The hook fails the build if the
  page it would strip is not the expected one.
- /zap.md: the agent-facing Markdown route now skips unlisted pages, and
  Head.astro no longer emits a `rel="alternate"` link pointing at it.
- sitemap: `@astrojs/sitemap` is configured explicitly (now a direct
  dependency) with a filter, which stops Starlight adding its own.

Page fixes: prettier formatting, second-person phrasing and sentence-case
headings for Vale ("Zap" added to the heading exceptions), GitHub casing,
h1 headings demoted below the page title, the visible iOS TODO removed,
and the screenshots moved to src/assets with descriptive names.
Checked against the Zap app source: the app is named "Shorebird Zap", the
floating button reads "Field note"/"Note" rather than "Notes", and its panel
says "Back to projects". The session button is "Start Session", the markup
button is "Mark up", sign-in is "Sign in", and the escape gesture is a
three-finger hold of about a second.
@AbhishekDoshi26
AbhishekDoshi26 merged commit c5dab03 into main Sep 19, 2026
4 checks passed
@AbhishekDoshi26
AbhishekDoshi26 deleted the docs/zap branch September 19, 2026 09:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants