Add progressive-disclosure build123d skill layer#18
Merged
Conversation
Skills live as skills/<name>/SKILL.md (pi/agentskills.io convention) with executable snippets, bundled into the client via raw glob imports. The system prompt carries only the catalog (pi formatSkillsForSystemPrompt); a new load_skill(name, resource?) tool pulls full bodies and referenced resource files on demand. Loads dedupe against the transcript and stay pinned across compaction by re-injection after the summary, keeping the compaction cut free to move. A deterministic verify-gate nudge points at the mapped skill on the second matching failure of a turn. Ablation treatments become none/core/catalog/full, with catalog the default and full (all bodies inlined) the preloaded upper bound. The sweep-and-loft seed skill ships with three snippets executed against real build123d by py-tests/test_skill_snippets.py. Also fixes the fake LLM title branch, which matched any system prompt containing the substring "title" and hijacked every e2e scenario once the prompt gained the phrase "titled results".
TypeBox Type.Tuple emits draft-07 tuple syntax (items as an array plus additionalItems), which the Anthropic API now rejects with "input_schema: JSON schema is invalid. It must match JSON Schema draft 2020-12". That made every live update_plan-bearing request fail with a 400. Replace the tuples with fixed-length homogeneous arrays, which validate identically for these coordinate and range fields. Caught by the live smoke session for the skill layer; the fake LLM never validates schemas, so no scripted test could see it.
SmartAI
added a commit
that referenced
this pull request
Jul 12, 2026
Pure content on the #18 mechanism: placement-and-frames, booleans-and-features, selectors-and-patterns, holes-and-threads, and surgical-edits, each with the fixed template and executable snippets. All snippets run against real build123d in py-tests, asserting volume deltas, solid counts, and topology, including a measurable demo of the Hole-drills-both-directions gotcha and a surgical edit whose collateral alarms are the assertions. The verify-gate nudge table gains four conservative mappings (first match wins); surgical-edits stays unmapped because a regression is not recognizable from a single failure text.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Adds a progressive-disclosure skill layer that teaches the agent build123d modeling technique beyond what doc search can provide, plus the ablation framework to measure it.
Three tiers of disclosure:
formatSkillsForSystemPrompt, plus the core invariant summary.load_skill(name)tool returns the fullSKILL.md, wrapped in pi'sformatSkillInvocationblock.load_skill(name, resource)returns files the skill references by relative path (e.g. a diagnostic probe script).Mechanism
packages/client/src/agent/skills/<name>/SKILL.md(pi/agentskills.io frontmatter convention) withsnippets/*.py, bundled viaimport.meta.glob+?raw.{{snippet:...}}markers interpolate snippet files into the body, so every inline code example is the exact bytes thatpy-tests/test_skill_snippets.pyexecutes against real build123d.Skill hint: load_skill("...")line to the failing run result; suppressed once loaded, silent for unmapped failures.none/core/catalog(default) /full, wherefullinlines every skill body as the preloaded upper bound that the pull-based catalog arm is measured against. The eval runner accepts the new conditions and hashes the whole skills tree.Seed skill
sweep-and-loft: profile framing (Plane(origin=path @ 0, z_dir=path % 0)), bend-radius and path-connectivity invariants, loft section ordering, failure signatures, and three executable snippets (sweep recipe, loft recipe, diagnostic probe resource). The remaining five planned skills land in a follow-up PR.Fixes along the way
TITLE_SYSTEM_PROMPTexactly.Validation
npm run typecheck,npm test(399 tests),npm run buildgreen.py-tests/(148 tests) green, including the new snippet suite that executes all three skill snippets against real build123d 0.11.1.e2e/skill-loop.spec.ts(fake LLM): load -> resource fetch -> dedupe notice, asserted on the persisted transcript.Human validation
A live smoke session (real LLM, sweep-heavy prompt) is the next step after merge to confirm the agent voluntarily loads from the catalog; the full ablation matrix (4 conditions x N trials) runs after that via
packages/client/eval/run-build123d-ablation.mjs.🤖 Generated with Claude Code