Skip to content

TypeAgent Studio (part 5): two-mode Impact Report replay#2561

Closed
TalZaccai wants to merge 1 commit into
dev/talzacc/typeagent_studio_part4from
dev/talzacc/typeagent_studio_part5
Closed

TypeAgent Studio (part 5): two-mode Impact Report replay#2561
TalZaccai wants to merge 1 commit into
dev/talzacc/typeagent_studio_part4from
dev/talzacc/typeagent_studio_part5

Conversation

@TalZaccai

Copy link
Copy Markdown
Contributor

Summary

Stacked on #2553 (part 4). Makes the Impact Report replay's deterministic dispatch model explicit instead of implicitly mixing two paths that never coexist in a real dispatcher config.

Two modes, surfaced as a toggle in the report's action bar:

  • nfa-grammar (default) — both versions match against the compiled grammar only; the construction cache is never consulted, so A and B stay symmetric. The cleanest signal for what a grammar/schema edit changed. Faithful to the dispatcher's nfa grammar system.
  • completionBased-cache (opt-in) — the working-tree side consults the live construction cache before the grammar (the default-dispatcher path). A B-side cache hit reflects what the dispatcher would serve, but the run is asymmetric.

Changes

Level A — core gating

  • Add StudioReplayMode = "nfa-grammar" | "completionBased-cache" and mode? on StudioReplayRequest (default nfa-grammar).
  • Gate the live construction-cache consult behind completionBased-cache. Default runs are now grammar-only and A/B-symmetric; the construction-cache method label no longer auto-appears.

Level B — plumbing + UI + test

  • Thread mode through the webview run message; parseWebviewMessage validates it (unknown/missing → nfa-grammar). The host forwards it into replayCorpus; the channel/RPC layers ride on StudioReplayRequest unchanged.
  • Add a two-state Grammar ⇄ Cache toggle to the action bar with explanatory tooltips, persisted/restored alongside the version selection and disabled with the other controls during a run.
  • Add an injectable resolveConstructionCache seam to CreateStudioRuntimeOptions (mirrors replayResolver/collisionScanner) so the gating is testable without a live cache.

Testing

  • pnpm run build typeagent-core — green
  • Studio suite — 184/184
  • Core replay specs (replay|grammarReplay|constructionCache) — 40/40
  • 3 new runtime gating tests: cache skipped in nfa-grammar (and when mode omitted), consulted in completionBased-cache
  • esbuild webview bundle clean; prettier clean

Notes

  • grammarResolver.ts / constructionCacheResolver.ts unchanged — the gate lives at the runtime call site.
  • Next fidelity rung (separate slice): L4a — stand up one live working-tree agent and run real validateWildcardMatch/validateEntityWildcardMatch.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

…ache)

Make the replay's deterministic dispatch model explicit instead of implicitly
mixing two paths that never coexist in a real dispatcher config.

Level A (core gating): add StudioReplayMode = "nfa-grammar" |
"completionBased-cache" and mode? on StudioReplayRequest (default
nfa-grammar); gate the live construction-cache consult behind
completionBased-cache. Default runs are now grammar-only and A/B-symmetric.

Level B (plumbing + UI + test):
- Thread mode through the webview run message; parseWebviewMessage validates
  it (unknown/missing -> nfa-grammar). The host forwards it into replayCorpus;
  the channel/RPC layers ride on StudioReplayRequest unchanged.
- Add a two-state Grammar/Cache toggle to the Impact Report action bar with
  explanatory tooltips, persisted/restored with the version selection.
- Add an injectable resolveConstructionCache seam to CreateStudioRuntimeOptions
  so the gating is testable without a live cache.
- Tests: 3 runtime gating tests over a scaffolded agent (cache skipped in
  nfa-grammar / when mode omitted, consulted in completionBased-cache); update
  webview protocol run-message expectations to include mode.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@TalZaccai TalZaccai changed the base branch from main to dev/talzacc/typeagent_studio_part4 June 25, 2026 23:32
@TalZaccai TalZaccai closed this Jun 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant