Skip to content

Plan artifact: evidence-checked multi-component design plans#13

Merged
SmartAI merged 5 commits into
mainfrom
claude/plan-model-design-7c7054
Jul 12, 2026
Merged

Plan artifact: evidence-checked multi-component design plans#13
SmartAI merged 5 commits into
mainfrom
claude/plan-model-design-7c7054

Conversation

@SmartAI

@SmartAI SmartAI commented Jul 11, 2026

Copy link
Copy Markdown
Owner

What

The agent can now maintain a plan of record for multi-component designs, enforced by the loop rather than trusted on prose:

  • update_plan tool — full-snapshot plan (goal, components, interfaces). Validation: unique label-safe ids, every component held by an interface (or an explicit free-floating reason) with a connected interface graph, no silent drops (removing unfinished work requires a stated abandon reason), and evidence-checked done: a component can only be marked done after a gate-passed run declared it via the new COMPONENT block and ran its planned checks verbatim.
  • Required volume check per component — every buildable component must carry {"kind": "volume", "range_mm3": [lo, hi], "target": id} with hi <= 1.5x lo. Volume is the cheapest topology detector; deriving it from intent before building is what actually catches sealed cavities and eaten floors (see Why).
  • Loop enforcement — stopping with unfinished components injects one deterministic follow-up (never twice without an intervening run); a plan with interfaces is unfinished until one gate-passed run declares all components together (where clearances are actually measured); with an active plan the CAD budget is per component bucket under a 3x global ceiling.
  • Probe runsCOMPONENT = "probe" marks diagnostics: they never advance the plan, never displace the deliverable in the viewer/script/artifact store, never become the conversation's verification badge, and only drain the global ceiling.
  • HarnessCOMPONENT block parse + echo, target on hole checks (per-child census), new hole_internal kind, max_mm on clearance (touching is now assertable), and a floating flag on children with no touching/interpenetrating partner.
  • UI — live plan card (components, statuses, abandon reasons, interfaces) and a plan-check chip for injected nudges.

Simple single-part requests are untouched: no plan, byte-identical behavior.

Why

Live stress tests of complex assemblies showed the loop building well but verifying blind: a gearbox base whose "open top" was accidentally sealed by a full-slab flange passed the gate, all self-authored checks, and every view (the lid occluded it), and the agent confidently reported it correct. A plate-stack run also shipped a top plate touching nothing.

The plan artifact closes this structurally: per-component builds are verified where defects are visible, progress claims require gate evidence, interfaces are first-class and must be measured, and the volume requirement forces the agent to do the arithmetic its feature-level checks skip. In live verification the same prompt that previously produced the sealed housing now yields an open one — the agent derives the expected volume first and models accordingly, and the measured volumes match the derivation exactly.

Notes for review

  • Plan snapshots live in the update_plan tool result (details.plan), not a custom message role: tool results are already persisted/replayed, and the compaction cut now never passes the newest accepted plan. Deviation rationale is in the commit messages.
  • Budget attribution parses COMPONENT client-side (the abort happens before the harness runs); the harness echo remains the only trust anchor for evidence — a lying declaration can waste budget but cannot fake progress.
  • _resolve_target also matches the result's own label so a solo component run satisfies the same targeted checks as the assembly run (the evidence rule requires byte-identical checks in both).
  • py-tests: 24 new tests in test_plan_component.py; one pinned normalization expectation in test_checks.py updated (hole specs now carry target). No golden.json changes (new measurement keys are additive).
  • Tested: 311 vitest + 128 py-tests green, typecheck green, e2e battery green including the new plan-flow.spec.ts (fake LLM); live-verified end-to-end with a real model on the gearbox assembly.

🤖 Generated with Claude Code

SmartAI added 5 commits July 11, 2026 16:17
The agent can now maintain a plan of record for multi-component designs:
a full-snapshot update_plan tool validated against the transcript (unique
label-safe ids, interface coverage + connected graph, no silent drops,
evidence-checked done transitions), persisted in the tool result and kept
verbatim across compaction.

Loop enforcement: stopping with unfinished components injects one
deterministic follow-up (never twice without an intervening run); with an
active plan the CAD budget is per component bucket under a 3x global
ceiling, and probe runs (COMPONENT = "probe") drain only the ceiling.

Probe runs also stop displacing the deliverable: they no longer publish
to the viewer, script panel, or artifact store, and never become the
conversation's verification badge.

Prompt gains the planning discipline (plan-first trigger, evidence-first
done, physical-coherence rule) and the UI renders the live plan card plus
a plan-check chip for injected nudges.
…stic

COMPONENT = "lid" (or a list; "probe" for diagnostics) is parsed from the
script and echoed in measurements together with the raw CHECKS entries -
the evidence link update_plan uses to accept done transitions. A malformed
declaration fails the gate as component_block.

CHECKS vocabulary grows what per-component plan gates need: hole_through/
hole_blind accept a target child (census runs on that child alone, so a
pin-occupied bore still classifies by the component's own geometry), a new
hole_internal kind counts buried bores, and clearance takes max_mm so
touching (max 0) and controlled fits become assertable.

Clearance diagnostics now flag floating children: a child whose every
pairwise state is apart is listed under measurements.floating - the
unsupported-part defect the plate-stack stress test shipped undetected.
…nce stop-gate

The first live re-run proved the machinery works (plan-first, per-component
gates, evidence-checked done, voluntary assembly run) and also that it was
not enough: the agent re-modeled the same sealed "open-top" cavity and its
planned checks (holes, bbox, symmetry) were structurally blind to it -
check authorship shares the geometry author's blind spot.

Two structural answers:
- update_plan now requires every buildable component to carry a volume
  check targeting itself, with range_mm3 no looser than 1.5x. The expected
  volume is derived from the component's INTENT (open top -> open volume),
  so wrong topology fails numerically. Loose ranges are rejected.
- A plan with interfaces only counts as finished once one gate-passed run
  declared ALL components together (where clearances are actually
  measured); stopping without that draws an assembly nudge. The live run
  hit exactly this window: both components done, interfaces unverified.

Fake plan-flow scenario and fixtures updated accordingly; e2e green.
Codex review (P2 on PR #13): hasAssemblyEvidence accepted any gate-passed
run that merely declared all component ids, so an assembly script that
omitted the planned clearance checks satisfied the stop-gate with the
interfaces never measured - the same authorship blind spot the component
evidence rule closes, left open for interfaces.

The all-components run's echoed CHECKS must now contain each clearance
interface's entry (either endpoint order; bounds must match the plan).
Captive interfaces compile to no measurable check and are carried by the
all-components run itself. Interfaces with an abandoned endpoint are
ignored. Prompt states the assembly script must include the interface
entries verbatim.
@SmartAI SmartAI merged commit 27642ca into main Jul 12, 2026
1 check passed
@SmartAI SmartAI deleted the claude/plan-model-design-7c7054 branch July 12, 2026 00:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant