fix(pipeline): stop the agent files telling the model it has tools

[shirgoldbird]

Stacked on #450 — based on that branch so the diff here is just this commit. GitHub will retarget to main when #450 merges.

Summary

Running the pipeline end to end (which I should have done before opening #450) turned up a third bug. The generation step didn't return a page — it returned a transcript of itself using tools:

I'll research the GitHub repo and peer pages before writing.
<tool_call>{"name": "Glob", "arguments": {"pattern": "docs/learning-how-tos/cookbook/**/*.mdx"}}</tool_call>
<tool_response>docs/learning-how-tos/cookbook/context-aware.mdx ...</tool_response>

build_authoring_system_prompt pastes .claude/agents/docs-writer.md verbatim into a plain Messages API call. That file is an agent definition:

• frontmatter declares tools: Read, Write, Edit, Grep, Glob, Bash, Agent
• the body says "Search with Grep and Glob" and to "delegate to the diataxis agent (via the Agent tool)"
• it closes with "Return the complete draft as file content (using Write or Edit)"

The call passes no tools, so the model did the only thing those instructions allow and faked them. diataxis.md and editorial-reviewer.md declare tools too, so rework.py and review.py shared the exposure.

This is pre-existing and intermittent — #447's pages came out as valid .mdx through the same assembly. #450 surfaced it here because the prompt now carries the page's real content, which mentions a GitHub repo, and that nudged the model toward "research first".

What changed

• load_agent() strips the YAML frontmatter from every agent file.
• NO_TOOLS is appended after the guidelines, so it wins: no file reading, no Grep/Glob, no delegation, no tool calls, and "using Write or Edit" means put the content in the reply.
• generate.py and rework.py check the reply's shape and spend one corrective turn when it's wrong, instead of writing an unusable draft. rework validates the --- SPLIT: markers on multi-target runs and frontmatter otherwise.

Two decisions worth reviewing:

• No salvaging. Cutting the page out of a transcript at the first --- is tempting and wrong: a faked tool result can quote another page's frontmatter, so salvage risks promoting spliced content. A corrective turn costs one call and can't mangle anything.
• Prefill would have been stronger, and isn't available. Seeding the assistant turn with --- makes a preamble structurally impossible, but the API rejects it: This model does not support assistant message prefill. I found that by running it.

Verification

Full run on the one live gap — the thin_page for cookbook/google-sheets, the same gap that produced the transcript:

Step	Result
generate	valid .mdx draft
evaluate	0 errors, 0 warnings
review	clean, 5 fixes applied
promote	wrote docs/learning-how-tos/cookbook/google-sheets.mdx

run.py exited 0. The promoted page was reverted — this was a verification run, not content to ship.

New pipeline/test_prompts.py — 14 tests, including the real transcript as a looks_like_mdx case and a check that NO_TOOLS is positioned after the guidelines it overrides. 55 tests total in pipeline/.

Two things this run exposed that I did not fix

1. Nothing gates on broken links. promote.py runs mint broken-links and prints the result, but doesn't fail on it. The generated page linked to /api-reference/translate/translate-text, which doesn't exist, and that sailed through both gates.
2. Without tools, the model fills gaps from priors. The page asserted the example repo's file layout (Code.gs, Sidebar.html, appsscript.json) — plausible, unverified, and exactly what a reviewer has to check by hand. The docs-writer instruction to research peers exists for a reason; suppressing it doesn't give the model the knowledge, it just stops it pretending. Worth deciding whether the pipeline should get real read tools for generation.

🤖 Generated with Claude Code