diff --git a/AGENTS.md b/AGENTS.md index a533372..89bf1b4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,7 +1,7 @@ # threadle — agent instructions threadle is a local-first node-graph patchbay for CLI agent sessions (Claude Code, -opencode, Cursor Agent, Antigravity, Codex, GitHub Copilot, Grok Build): observability (blueprints, activity, statistics) plus context +opencode, Cursor Agent, Antigravity, Codex, GitHub Copilot, Grok Build, Muse Code): observability (blueprints, activity, statistics) plus context wiring/handoff between sessions on a canvas. ## Vocabulary (UI + docs) @@ -34,7 +34,7 @@ Plain terms lead. Graph JSON keeps historical type ids for compatibility. and the built-in `NodeDefinition` catalog (`src/nodes/`). Single source of truth; UI and server both import it. - `packages/server` — Hono API (`bin: threadle`). Providers under - `src/providers/{claude-code,opencode,cursor,antigravity,codex,copilot,grok}` read agent storage **read-only**; + `src/providers/{claude-code,opencode,cursor,antigravity,codex,copilot,grok,muse}` read agent storage **read-only**; all runs/injections go through each tool's CLI or HTTP surface. - `packages/web` — Vue 3 + Vue Flow. True-black monochrome theme (`src/theme/theme.css`); no emojis, text glyphs only (❯ ⎇ ⟨/⟩ ▤ ⚙ ✦). @@ -49,7 +49,7 @@ Plain terms lead. Graph JSON keeps historical type ids for compatibility. custom nodes / stdlib / recipes. Shared pure helpers live in `packages/shared` (`judge`, `until`, `wait-idle`, `iterator`, …). - Never write into `~/.claude/projects`, opencode's SQLite, `~/.cursor` - chats/transcripts, `~/.gemini/antigravity-cli`, `~/.codex`, `~/.copilot`, or `~/.grok`; parsers must skip unknown + chats/transcripts, `~/.gemini/antigravity-cli`, `~/.codex`, `~/.copilot`, `~/.grok`, `~/.muse`, or `~/.local/share/muse`; parsers must skip unknown record types, not throw (formats are undocumented and churn). - All list/table UIs: mono micro-labels, right-aligned numeric columns, `minmax(0, 1fr)` name columns, values ellipsize instead of wrapping. diff --git a/README.md b/README.md index 334051b..3904594 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ release

-Your agents write everything down: transcripts, tool calls, files touched, spend. Then nobody reads it, and the context dies at each tool boundary. threadle reads it, draws it, and lets you wire it into the next run. Every [Claude Code](https://claude.com/claude-code), [opencode](https://opencode.ai), [Cursor](https://cursor.com) `agent`, [Antigravity](https://antigravity.google) `agy`, [Codex](https://github.com/openai/codex), [GitHub Copilot](https://github.com/features/copilot) CLI, and [Grok Build](https://x.ai/cli) session becomes a wireable node. Inspect what happened. Hand context across tools. Run the pipeline. +Your agents write everything down: transcripts, tool calls, files touched, spend. Then nobody reads it, and the context dies at each tool boundary. threadle reads it, draws it, and lets you wire it into the next run. Every [Claude Code](https://claude.com/claude-code), [opencode](https://opencode.ai), [Cursor](https://cursor.com) `agent`, [Antigravity](https://antigravity.google) `agy`, [Codex](https://github.com/openai/codex), [GitHub Copilot](https://github.com/features/copilot) CLI, [Grok Build](https://x.ai/cli), and [Muse Code](https://dev.meta.ai) session becomes a wireable node. Inspect what happened. Hand context across tools. Run the pipeline. **Yours, with no catch.** Discovery is read-only against storage you already have. threadle binds `127.0.0.1`, has no account and no telemetry, and does not upload transcripts. Pressing ▶ drives each tool’s own CLI under your credentials. If threadle disappears tomorrow, your sessions are exactly where they always were. @@ -118,7 +118,7 @@ The full set lives in [`docs/screenshots/`](https://github.com/threadle-sh/threa ## Trust -Parsers never mutate `~/.claude`, opencode’s SQLite, `~/.cursor`, `~/.gemini/antigravity-cli`, `~/.codex`, `~/.copilot`, or `~/.grok`. Runs spawn the official CLI, and *it* writes new sessions. +Parsers never mutate `~/.claude`, opencode’s SQLite, `~/.cursor`, `~/.gemini/antigravity-cli`, `~/.codex`, `~/.copilot`, or `~/.grok`, `~/.muse`, or `~/.local/share/muse`. Runs spawn the official CLI, and *it* writes new sessions. The server rejects foreign `Host` (DNS rebinding) and cross-site write `Origin` (CSRF). There is no remote mode and no auth token. List prices ship bundled (CI-refreshed from [models.dev](https://models.dev)). The process does not fetch them at runtime. Everything threadle persists is under `~/.config/threadle/`. diff --git a/docs/cli/manual.md b/docs/cli/manual.md index 6238095..e86244d 100644 --- a/docs/cli/manual.md +++ b/docs/cli/manual.md @@ -515,7 +515,7 @@ UI **▶** (no gates) and **≫** use the same `POST /api/run/workflow` path as | `no skill named "…" found` | Install/unmute the skill, or mute the node in the UI and re-export | | `nothing to run for "…"` | Path, recipe id, template id, or graph id wrong; run `threadle recipes` / `--list-templates` | | `threadle server not reachable` | Start `threadle --no-open` before `jobs` / `services` / `skills` / `open` / `run --detach` | -| Agent CLI not found | Install Cursor `agent` / Claude Code / opencode / Antigravity `agy` / Codex / GitHub Copilot / Grok Build; ensure on `PATH` — or run `threadle check` | +| Agent CLI not found | Install Cursor `agent` / Claude Code / opencode / Antigravity `agy` / Codex / GitHub Copilot / Grok Build / Muse Code; ensure on `PATH` — or run `threadle check` | | Wrong project files | Pass `--dir` to the repo root you intend | | `context not materialized` / blocked ❝ | Wire a session/agent (extracts mid-run), drag from Library, or mute | diff --git a/docs/provider-freshness.md b/docs/provider-freshness.md index ba89ce2..317d85b 100644 --- a/docs/provider-freshness.md +++ b/docs/provider-freshness.md @@ -1,6 +1,6 @@ # Keeping providers fresh -Agent CLIs (Claude Code, Cursor `agent`, opencode, Antigravity `agy`, Codex, GitHub Copilot, Grok Build) store transcripts in **undocumented** formats and rename flags without notice. threadle does **not** mirror their configs/APIs with a daily diff bot. +Agent CLIs (Claude Code, Cursor `agent`, opencode, Antigravity `agy`, Codex, GitHub Copilot, Grok Build, Muse Code) store transcripts in **undocumented** formats and rename flags without notice. threadle does **not** mirror their configs/APIs with a daily diff bot. ## Strategy @@ -33,7 +33,7 @@ It does **not** parse transcripts or prove schemas (that’s fixture CI). ### What `--providers` adds -1. **Golden fixtures** (when `packages/server/test/fixtures/providers` is on disk — repo checkout / CI): parse one sample per provider (`fixture:claude-code`, `fixture:cursor`, … `fixture:grok`). Published installs skip these rows. +1. **Golden fixtures** (when `packages/server/test/fixtures/providers` is on disk — repo checkout / CI): parse one sample per provider (`fixture:claude-code`, `fixture:cursor`, … `fixture:grok`, `fixture:muse`). Published installs skip these rows. 2. **Inject flag probes** for each **installed** CLI — run help and look for inject-critical tokens listed in [`packages/server/src/providers/freshness/inject-flags.ts`](../packages/server/src/providers/freshness/inject-flags.ts): | Detail | Behavior | @@ -41,13 +41,15 @@ It does **not** parse transcripts or prove schemas (that’s fixture CI). | Missing binary | Skip (`ok`) | | Help text contains any `match` string | `ok` | | No match | `!!` and non-zero exit | -| Subcommand help | Some probes use e.g. `opencode run --help` (not only top-level `--help`) | +| Subcommand help | Some probes use e.g. `opencode run --help` or `muse exec --help` (not only top-level `--help`) | | Bracket notation | Claude documents `--append-system-prompt[-file]` — both that form and the long flag count | | Fixture missing | Skip (`ok`) | | Fixture parse / shape fail | `!!` and non-zero exit | A `!! flag:` row means “help no longer mentions a string we rely on for inject/run” — confirm with the real CLI before changing adapters (help text and the live flag can diverge). A `!! fixture:` row means a golden sample stopped normalizing. +**Muse models.** Upstream has no `muse models` list yet. threadle’s model picker scrapes Spark ids from `muse exec --help` / `muse --help`, keeps hardcoded fallbacks, and honors `MUSE_MODELS`. That catalog is soft — expand fallbacks or env when Meta renames Spark ids. + ## Adding a fixture 1. Anonymize one real transcript line (paths → `/tmp/…`, strip secrets). diff --git a/docs/screenshots/canvas.png b/docs/screenshots/canvas.png index 8f0a32b..6c9a44f 100644 Binary files a/docs/screenshots/canvas.png and b/docs/screenshots/canvas.png differ diff --git a/docs/screenshots/sessions.png b/docs/screenshots/sessions.png index 1ce15aa..b09f42d 100644 Binary files a/docs/screenshots/sessions.png and b/docs/screenshots/sessions.png differ diff --git a/docs/screenshots/statistics.png b/docs/screenshots/statistics.png index 964dea8..600df14 100644 Binary files a/docs/screenshots/statistics.png and b/docs/screenshots/statistics.png differ diff --git a/docs/screenshots/stats.png b/docs/screenshots/stats.png new file mode 100644 index 0000000..600df14 Binary files /dev/null and b/docs/screenshots/stats.png differ diff --git a/package-lock.json b/package-lock.json index eb6f7ae..06eba3f 100644 --- a/package-lock.json +++ b/package-lock.json @@ -6935,7 +6935,7 @@ }, "packages/server": { "name": "threadle", - "version": "1.0.10", + "version": "1.1.1", "license": "MIT", "dependencies": { "@hono/node-server": "^1.13.7", @@ -6966,14 +6966,14 @@ }, "packages/shared": { "name": "@threadle/shared", - "version": "1.0.10", + "version": "1.1.1", "dependencies": { "zod": "^4.6.5" } }, "packages/web": { "name": "@threadle/web", - "version": "1.0.10", + "version": "1.1.1", "dependencies": { "@threadle/shared": "*", "@vue-flow/background": "^1.3.2", diff --git a/packages/server/package.json b/packages/server/package.json index 5d3850b..43caa78 100644 --- a/packages/server/package.json +++ b/packages/server/package.json @@ -1,7 +1,7 @@ { "name": "threadle", - "version": "1.0.11", - "description": "Local-first node-graph patchbay for Claude Code, opencode, Cursor Agent, Antigravity, Codex, Copilot, and Grok Build sessions.", + "version": "1.1.1", + "description": "Local-first node-graph patchbay for Claude Code, opencode, Cursor Agent, Antigravity, Codex, Copilot, Grok Build and Muse sessions.", "type": "module", "license": "MIT", "author": "Fabian Bienk (https://github.com/zFarbp)", @@ -22,6 +22,7 @@ "codex", "copilot", "grok", + "muse", "agent", "workflow", "patchbay" diff --git a/packages/server/src/check.ts b/packages/server/src/check.ts index f577c74..625ec1c 100644 --- a/packages/server/src/check.ts +++ b/packages/server/src/check.ts @@ -12,6 +12,7 @@ import { probeInjectFlags } from "./providers/freshness/probe.js"; import { probeGoldenFixtures } from "./providers/freshness/fixtures.js"; import { probeUpstream } from "./providers/freshness/upstream.js"; import { grokBin } from "./providers/grok/paths.js"; +import { museBin } from "./providers/muse/paths.js"; import { codexBin } from "./providers/codex/paths.js"; import { copilotBin } from "./providers/copilot/paths.js"; @@ -154,6 +155,7 @@ export async function runCheck( checkPreferredOrPath("cli:codex", codexBin(), "codex"), checkPreferredOrPath("cli:copilot", copilotBin(), "copilot"), checkPreferredOrPath("cli:grok", grokBin(), "grok"), + checkPreferredOrPath("cli:muse", museBin(), "muse"), ]); checks.push(...cliChecks); diff --git a/packages/server/src/jobs.ts b/packages/server/src/jobs.ts index 8da6503..0d15b47 100644 --- a/packages/server/src/jobs.ts +++ b/packages/server/src/jobs.ts @@ -19,6 +19,7 @@ export interface JobRecord { label?: string; /** workflow (graph) that launched the run, when known */ graphId?: string; + sessionRef?: { provider: string; sessionId: string }; status: JobStatus; createdAt: number; finishedAt?: number; @@ -48,7 +49,10 @@ class JobRegistry extends EventEmitter { kind: JobKind, label?: string, graphId?: string, - opts?: { clientDriven?: boolean }, + opts?: { + clientDriven?: boolean; + sessionRef?: { provider: string; sessionId: string }; + }, ): { jobId: string; signal: AbortSignal } { this.counter += 1; const jobId = `job_${Date.now().toString(36)}_${this.counter}`; @@ -59,6 +63,7 @@ class JobRegistry extends EventEmitter { kind, label, graphId, + sessionRef: opts?.sessionRef, status: "running", createdAt: Date.now(), touchedAt: Date.now(), diff --git a/packages/server/src/pilot-sessions/store.ts b/packages/server/src/pilot-sessions/store.ts new file mode 100644 index 0000000..eeae56e --- /dev/null +++ b/packages/server/src/pilot-sessions/store.ts @@ -0,0 +1,108 @@ +import fs from "node:fs"; +import path from "node:path"; +import { threadleConfigDir } from "../paths.js"; + +export interface PilotSessionEntry { + provider: string; + sessionId: string; + markedAt: number; + caseId?: string; + tokensIn?: number; + tokensOut?: number; + promptEst?: number; + baselineEst?: number; + baselineNote?: string; + ignoreLocalMarkdown?: boolean; +} + +interface PilotSessionsFile { + version: 1; + entries: PilotSessionEntry[]; +} + +function storePath(): string { + return path.join(threadleConfigDir(), "pilot-sessions.json"); +} + +export function pilotSessionKey(provider: string, sessionId: string): string { + return `${provider}:${sessionId}`; +} + +async function readFile(): Promise { + try { + const raw = JSON.parse( + await fs.promises.readFile(storePath(), "utf8"), + ) as PilotSessionsFile; + if (raw?.version === 1 && Array.isArray(raw.entries)) return raw; + } catch { + // missing / corrupt -> empty + } + return { version: 1, entries: [] }; +} + +async function writeFile(data: PilotSessionsFile): Promise { + await fs.promises.mkdir(threadleConfigDir(), { recursive: true }); + await fs.promises.writeFile(storePath(), JSON.stringify(data, null, 2), "utf8"); +} + +export async function markPilotSession(opts: { + provider: string; + sessionId: string; + caseId?: string; + tokensIn?: number; + tokensOut?: number; + promptEst?: number; + baselineEst?: number; + baselineNote?: string; + ignoreLocalMarkdown?: boolean; +}): Promise { + const data = await readFile(); + const key = pilotSessionKey(opts.provider, opts.sessionId); + const existing = data.entries.find( + (e) => pilotSessionKey(e.provider, e.sessionId) === key, + ); + if (existing) { + if (opts.caseId && !existing.caseId) existing.caseId = opts.caseId; + if (opts.tokensIn != null) existing.tokensIn = opts.tokensIn; + if (opts.tokensOut != null) existing.tokensOut = opts.tokensOut; + if (opts.promptEst != null) existing.promptEst = opts.promptEst; + if (opts.baselineEst != null) existing.baselineEst = opts.baselineEst; + if (opts.baselineNote) existing.baselineNote = opts.baselineNote; + if (opts.ignoreLocalMarkdown != null) { + existing.ignoreLocalMarkdown = opts.ignoreLocalMarkdown; + } + await writeFile(data); + return existing; + } + const entry: PilotSessionEntry = { + provider: opts.provider, + sessionId: opts.sessionId, + markedAt: Date.now(), + caseId: opts.caseId, + tokensIn: opts.tokensIn, + tokensOut: opts.tokensOut, + promptEst: opts.promptEst, + baselineEst: opts.baselineEst, + baselineNote: opts.baselineNote, + ignoreLocalMarkdown: opts.ignoreLocalMarkdown, + }; + data.entries.unshift(entry); + if (data.entries.length > 2000) data.entries.length = 2000; + await writeFile(data); + return entry; +} + +export async function readPilotSessionMap(): Promise> { + const data = await readFile(); + return new Map( + data.entries.map((e) => [pilotSessionKey(e.provider, e.sessionId), e] as const), + ); +} + +export async function readPilotSessionKeys(): Promise> { + return new Set((await readPilotSessionMap()).keys()); +} + +export async function listPilotSessions(): Promise { + return (await readFile()).entries; +} diff --git a/packages/server/src/providers/antigravity/discover.ts b/packages/server/src/providers/antigravity/discover.ts index 27df669..5dddf1f 100644 --- a/packages/server/src/providers/antigravity/discover.ts +++ b/packages/server/src/providers/antigravity/discover.ts @@ -300,7 +300,9 @@ export async function liveStatuses(): Promise> { const map = new Map(); const summaries = await loadSummaryIndex(); for (const [id, sum] of summaries) { - if (sum.status && sum.status !== "unknown") map.set(id, sum.status); + if (sum.status === "running" || sum.status === "waiting") { + map.set(id, sum.status); + } } return map; } diff --git a/packages/server/src/providers/antigravity/inject.ts b/packages/server/src/providers/antigravity/inject.ts index be14bde..d5d0eef 100644 --- a/packages/server/src/providers/antigravity/inject.ts +++ b/packages/server/src/providers/antigravity/inject.ts @@ -1,7 +1,9 @@ import { execa } from "execa"; import type { ContextPayload, InjectResult, InjectTarget } from "@threadle/shared"; -import { forEachLine, type LogLane, type LogSink } from "../stream.js"; +import { estimateTokenBaseline, formatBaselineSuffix } from "@threadle/shared"; +import type { LogSink } from "../stream.js"; import { positionalSafe } from "../argv-safe.js"; +import { resolveRunHarnessExtras, logHarnessExtrasLanes } from "../harness-extras.js"; import { agyBin } from "./agy-bin.js"; import { recordAntigravityUsage } from "./usage.js"; @@ -62,17 +64,30 @@ function applyAgentMode(args: string[], agent?: string): void { } } +/** Build argv for a headless `agy` run (without --output-format). Exported for tests. */ +export function buildAgyAgentArgv(opts: { + sessionId?: string; + model?: string; + agent?: string; + prompt: string; + harnessExtras?: boolean; +}): string[] { + return baseArgs(opts); +} + function baseArgs(opts: { sessionId?: string; model?: string; agent?: string; prompt: string; + harnessExtras?: boolean; }): string[] { - const args = ["-p", "--dangerously-skip-permissions"]; + // `-p` / `--print` takes the prompt as its value (not a bare flag + positional). + const args = ["-p", positionalSafe(opts.prompt), "--dangerously-skip-permissions"]; if (opts.sessionId) args.push("--conversation", opts.sessionId); if (opts.model) args.push("--model", opts.model); + if (opts.harnessExtras === false) args.push("--disable-slash-commands"); applyAgentMode(args, opts.agent); - args.push(positionalSafe(opts.prompt)); return args; } @@ -91,94 +106,45 @@ async function persistUsage( async function runAgyJson( args: string[], cwd: string, + signal?: AbortSignal, ): Promise<{ sessionId?: string; resultText?: string; isError: boolean; usage?: AgyJsonResult["usage"]; + stderr?: string; }> { - const { stdout } = await execa(agyBin(), [...args.slice(0, -1), "--output-format", "json", args[args.length - 1]!], { + const child = await execa(agyBin(), [...args, "--output-format", "json"], { cwd, timeout: INJECT_TIMEOUT_MS, stdin: "ignore", + cancelSignal: signal, + reject: false, }); - const evt = parseJsonResult(stdout); - const isError = Boolean(evt?.status && !/^SUCCESS$/i.test(evt.status)); + const evt = parseJsonResult(child.stdout || ""); + const resultText = + typeof evt?.response === "string" ? evt.response.trimEnd() : undefined; + const statusFail = Boolean(evt?.status && !/^SUCCESS$/i.test(evt.status)); + const exitFail = child.exitCode !== 0 && child.exitCode !== undefined; + const emptyFail = !resultText?.trim(); return { sessionId: evt?.conversation_id, - resultText: typeof evt?.response === "string" ? evt.response.trimEnd() : undefined, - isError, + resultText, + isError: statusFail || exitFail || emptyFail, usage: evt?.usage, + stderr: (child.stderr || "").trim() || undefined, }; } -function agyEventToLogs(line: string): Array<[LogLane, string]> { - const out: Array<[LogLane, string]> = []; - try { - const evt = JSON.parse(line) as Record; - if (typeof evt.response === "string" && evt.response) { - out.push(["text", evt.response]); - } - if (typeof evt.text === "string" && evt.text) { - out.push(["text", evt.text]); - } - if (typeof evt.content === "string" && evt.content && evt.type === "PLANNER_RESPONSE") { - out.push(["text", evt.content]); - } - } catch { - // ignore +function agyFailMessage( + prefix: string, + result: { resultText?: string; stderr?: string }, +): string { + const detail = [result.stderr, result.resultText].filter(Boolean).join("\n").trim(); + if (!result.resultText?.trim() && !detail) { + return `${prefix}: no response`; } - return out; -} - -async function runAgyStreaming( - args: string[], - cwd: string, - onLog: LogSink, - signal?: AbortSignal, -): Promise<{ - sessionId?: string; - resultText?: string; - isError: boolean; - usage?: AgyJsonResult["usage"]; -}> { - // insert output-format before the prompt (last arg) - const prompt = args[args.length - 1]!; - const head = args.slice(0, -1); - const child = execa( - agyBin(), - [...head, "--output-format", "stream-json", prompt], - { - cwd, - timeout: INJECT_TIMEOUT_MS, - stdin: "ignore", - cancelSignal: signal, - }, - ); - let result: AgyJsonResult | undefined; - let sessionId: string | undefined; - const chunks: string[] = []; - forEachLine(child.stdout, (line) => { - chunks.push(line); - try { - const evt = JSON.parse(line) as AgyJsonResult & { conversation_id?: string }; - if (typeof evt.conversation_id === "string") sessionId = evt.conversation_id; - if (evt.response !== undefined || evt.usage || evt.status) result = evt; - for (const [lane, text] of agyEventToLogs(line)) onLog(lane, text); - } catch { - // ignore - } - }); - await child; - const stdout = chunks.join("\n"); - const parsed = result ?? parseJsonResult(stdout); - if (parsed?.response) onLog("text", parsed.response); - return { - sessionId: parsed?.conversation_id ?? sessionId, - resultText: typeof parsed?.response === "string" ? parsed.response.trimEnd() : undefined, - isError: Boolean(parsed?.status && !/^SUCCESS$/i.test(parsed.status)), - usage: parsed?.usage, - }; + return detail ? `${prefix}: ${detail.slice(0, 400)}` : prefix; } export async function injectAntigravity( @@ -197,13 +163,19 @@ export async function injectAntigravity( prompt: text, }); const result = await runAgyJson(args, target.projectDir); - if (result.isError) throw new Error("antigravity continue inject failed"); + if (result.isError) throw new Error(agyFailMessage("antigravity continue inject failed", result)); const newSessionId = result.sessionId ?? target.sessionId; await persistUsage(newSessionId, result.usage); return { newSessionId, provider: "antigravity", resultText: result.resultText, + usage: result.usage + ? { + inputTokens: result.usage.input_tokens, + outputTokens: result.usage.output_tokens, + } + : undefined, }; } @@ -214,13 +186,19 @@ export async function injectAntigravity( prompt: text, }); const result = await runAgyJson(args, target.projectDir); - if (result.isError) throw new Error("antigravity new-session inject failed"); + if (result.isError) throw new Error(agyFailMessage("antigravity new-session inject failed", result)); if (!result.sessionId) throw new Error("antigravity new-session returned no conversation_id"); await persistUsage(result.sessionId, result.usage); return { newSessionId: result.sessionId, provider: "antigravity", resultText: result.resultText, + usage: result.usage + ? { + inputTokens: result.usage.input_tokens, + outputTokens: result.usage.output_tokens, + } + : undefined, }; } @@ -235,30 +213,61 @@ export interface RunAntigravityAgentOptions { sessionId?: string; onLog?: LogSink; signal?: AbortSignal; + harnessExtras?: boolean; + ignoreLocalMarkdown?: boolean; } export async function runAntigravityAgent( opts: RunAntigravityAgentOptions, ): Promise { + const harnessExtras = await resolveRunHarnessExtras(opts.harnessExtras); const args = baseArgs({ sessionId: opts.sessionId, model: opts.model, agent: opts.agent, prompt: opts.prompt, + harnessExtras, }); + if (opts.onLog) { + logHarnessExtrasLanes(opts.onLog, "antigravity", harnessExtras); + } - const result = opts.onLog - ? await runAgyStreaming(args, opts.projectDir, opts.onLog, opts.signal) - : await runAgyJson(args, opts.projectDir); + // Always use final JSON — stream-json + stdin ignore aborts mid-turn on current agy. + const result = await runAgyJson(args, opts.projectDir, opts.signal); - if (result.isError) throw new Error("antigravity agent run failed"); + if (result.isError) { + throw new Error(agyFailMessage("antigravity agent run failed", result)); + } const newSessionId = result.sessionId ?? opts.sessionId; if (!newSessionId) throw new Error("antigravity agent run returned no conversation_id"); await persistUsage(newSessionId, result.usage); + if (opts.onLog) { + if (result.resultText) opts.onLog("text", result.resultText); + if (result.usage) { + const inTok = result.usage.input_tokens ?? 0; + const outTok = result.usage.output_tokens ?? 0; + const think = result.usage.thinking_tokens ?? 0; + const parts = [`${inTok}→${outTok} tok`]; + if (think) parts.push(`${think} think`); + if (result.usage.total_tokens) parts.push(`${result.usage.total_tokens} total`); + const b = estimateTokenBaseline(inTok, opts.prompt, { + ignoreLocalMarkdown: opts.ignoreLocalMarkdown, + }); + const suf = b ? formatBaselineSuffix(b) : ""; + if (suf) parts.push(suf); + opts.onLog("meta", `■ ${parts.join(" · ")}`); + } + } return { newSessionId, provider: "antigravity", resultText: result.resultText, + usage: result.usage + ? { + inputTokens: result.usage.input_tokens, + outputTokens: result.usage.output_tokens, + } + : undefined, }; } diff --git a/packages/server/src/providers/bare-workspace.ts b/packages/server/src/providers/bare-workspace.ts new file mode 100644 index 0000000..3c3b96b --- /dev/null +++ b/packages/server/src/providers/bare-workspace.ts @@ -0,0 +1,14 @@ +import fs from "node:fs"; +import path from "node:path"; +import { threadleConfigDir } from "../paths.js"; + +/** + * Empty cwd for smoke / pilot runs that must not ingest project AGENTS.md, + * CLAUDE.md, .cursor/rules, etc. Created once under the threadle config dir + */ +export async function ensureBareWorkspace(): Promise { + const dir = path.join(threadleConfigDir(), "pilot-bare"); + await fs.promises.mkdir(dir, { recursive: true }); + // Keep it empty — no AGENTS.md / CLAUDE.md / .cursor / .git + return dir; +} diff --git a/packages/server/src/providers/claude-code/inject.ts b/packages/server/src/providers/claude-code/inject.ts index ff9f38c..41ed7bd 100644 --- a/packages/server/src/providers/claude-code/inject.ts +++ b/packages/server/src/providers/claude-code/inject.ts @@ -3,9 +3,15 @@ import path from "node:path"; import crypto from "node:crypto"; import { execa } from "execa"; import type { ContextPayload, InjectResult, InjectTarget } from "@threadle/shared"; -import { enrichUsageExhaustionError } from "@threadle/shared"; +import { + enrichUsageExhaustionError, + estimateTokenBaseline, + formatBaselineSuffix, +} from "@threadle/shared"; import { threadleConfigDir } from "../../graphs/store.js"; +import { resolveRunHarnessExtras, logHarnessExtrasLanes } from "../harness-extras.js"; import { forEachLine, toolSummary, type LogLane, type LogSink } from "../stream.js"; +import { formatDuration } from "../run-metrics.js"; const INJECT_TIMEOUT_MS = 10 * 60_000; @@ -32,6 +38,16 @@ interface ClaudeJsonResult { result?: string; is_error?: boolean; subtype?: string; + duration_ms?: number; + duration_api_ms?: number; + num_turns?: number; + total_cost_usd?: number; + usage?: { + input_tokens?: number; + output_tokens?: number; + cache_read_input_tokens?: number; + cache_creation_input_tokens?: number; + }; } async function runClaude( @@ -41,6 +57,7 @@ async function runClaude( const { stdout } = await execa("claude", args, { cwd, timeout: INJECT_TIMEOUT_MS, + stdin: "ignore", env: { ...process.env, CLAUDE_CODE_ENTRYPOINT: "threadle-inject" }, }); return JSON.parse(stdout) as ClaudeJsonResult; @@ -119,6 +136,10 @@ export interface RunClaudeAgentOptions { sessionId?: string; /** Claude --permission-mode (overrides Plan agent default when set) */ permissionMode?: string; + /** When false, --disable-slash-commands. Omit = Settings. */ + harnessExtras?: boolean; + /** Skip project/local CLAUDE.md (see buildClaudeAgentArgv). */ + ignoreLocalMarkdown?: boolean; /** live stream of the run's text/tool events */ onLog?: LogSink; signal?: AbortSignal; @@ -138,12 +159,28 @@ export function buildClaudeAgentArgv(opts: { permissionMode?: string; /** When true, skip Plan/Explore defaults (caller adds --append-system-prompt-file). */ hasAgentSystemPrompt?: boolean; + /** + * When false, pass `--disable-slash-commands` (skip slash skills). + * Do not use `--bare` — it drops claude.ai login (`apiKeySource: none`). + * Omit / true = Claude defaults. + */ + harnessExtras?: boolean; + /** Skip project/local CLAUDE.md via `--setting-sources user`. */ + ignoreLocalMarkdown?: boolean; }): string[] { const args = opts.sessionId ? ["--resume", opts.sessionId, "-p", opts.prompt] : ["--session-id", opts.freshSessionId ?? "SESSION", "-p", opts.prompt]; if (opts.model) args.push("--model", opts.model); + if (opts.harnessExtras === false) { + args.push("--disable-slash-commands"); + } + if (opts.ignoreLocalMarkdown) { + // Keep user settings (auth prefs); drop project + local CLAUDE.md / hooks. + args.push("--setting-sources", "user"); + } + if (opts.hasAgentSystemPrompt) { if (opts.permissionMode) args.push("--permission-mode", opts.permissionMode); } else if (opts.permissionMode) { @@ -189,6 +226,7 @@ async function runClaudeStreaming( cwd: string, onLog: LogSink, signal?: AbortSignal, + metaOpts?: { prompt?: string; ignoreLocalMarkdown?: boolean }, ): Promise { const child = execa( "claude", @@ -197,6 +235,7 @@ async function runClaudeStreaming( cwd, timeout: INJECT_TIMEOUT_MS, cancelSignal: signal, + stdin: "ignore", env: { ...process.env, CLAUDE_CODE_ENTRYPOINT: "threadle-inject" }, }, ); @@ -205,6 +244,7 @@ async function runClaudeStreaming( const evt = JSON.parse(line) as ClaudeJsonResult & { type?: string }; if (evt.type === "result") { result = evt; + onLog("meta", claudeResultMeta(evt, metaOpts)); return; } for (const [lane, text] of claudeEventToLogs(line)) onLog(lane, text); @@ -214,11 +254,43 @@ async function runClaudeStreaming( return result; } +function claudeResultMeta( + evt: ClaudeJsonResult, + opts?: { prompt?: string; ignoreLocalMarkdown?: boolean }, +): string { + const parts: string[] = []; + if (typeof evt.duration_ms === "number") parts.push(formatDuration(evt.duration_ms)); + if (typeof evt.duration_api_ms === "number") { + parts.push(`api ${formatDuration(evt.duration_api_ms)}`); + } + if (typeof evt.num_turns === "number") { + parts.push(`${evt.num_turns} turn${evt.num_turns === 1 ? "" : "s"}`); + } + const u = evt.usage; + if (u) { + const inTok = u.input_tokens ?? 0; + const outTok = u.output_tokens ?? 0; + if (inTok || outTok) parts.push(`${inTok}→${outTok} tok`); + if (opts?.prompt && inTok) { + const b = estimateTokenBaseline(inTok, opts.prompt, { + ignoreLocalMarkdown: opts.ignoreLocalMarkdown, + }); + const suf = b ? formatBaselineSuffix(b) : ""; + if (suf) parts.push(suf); + } + } + if (typeof evt.total_cost_usd === "number") { + parts.push(`$${evt.total_cost_usd.toFixed(4)}`); + } + return parts.length ? `■ ${parts.join(" · ")}` : "■ done"; +} + /** Run a prompt headlessly as a new Claude session, optionally under an agent definition. */ export async function runClaudeAgent( opts: RunClaudeAgentOptions, ): Promise { const newId = opts.sessionId ?? crypto.randomUUID(); + const harnessExtras = await resolveRunHarnessExtras(opts.harnessExtras); const args = buildClaudeAgentArgv({ agent: opts.agent, model: opts.model, @@ -227,7 +299,15 @@ export async function runClaudeAgent( freshSessionId: opts.sessionId ? undefined : newId, permissionMode: opts.permissionMode, hasAgentSystemPrompt: Boolean(opts.agentSystemPrompt), + harnessExtras, + ignoreLocalMarkdown: opts.ignoreLocalMarkdown, }); + if (opts.onLog) { + logHarnessExtrasLanes(opts.onLog, "claude-code", harnessExtras); + if (opts.ignoreLocalMarkdown) { + opts.onLog("meta", "ignore local md · --setting-sources user"); + } + } if (opts.agentSystemPrompt) { const dir = path.join(threadleConfigDir(), "tmp"); @@ -238,7 +318,10 @@ export async function runClaudeAgent( } const result = opts.onLog - ? await runClaudeStreaming(args, opts.projectDir, opts.onLog, opts.signal) + ? await runClaudeStreaming(args, opts.projectDir, opts.onLog, opts.signal, { + prompt: opts.prompt, + ignoreLocalMarkdown: opts.ignoreLocalMarkdown, + }) : await runClaude([...args, "--output-format", "json"], opts.projectDir); if (result.is_error) { throw enrichUsageExhaustionError( @@ -251,5 +334,11 @@ export async function runClaudeAgent( newSessionId: result.session_id ?? newId, provider: "claude-code", resultText: result.result, + usage: result.usage + ? { + inputTokens: result.usage.input_tokens, + outputTokens: result.usage.output_tokens, + } + : undefined, }; } diff --git a/packages/server/src/providers/codex/inject.ts b/packages/server/src/providers/codex/inject.ts index ea23917..b76b2b0 100644 --- a/packages/server/src/providers/codex/inject.ts +++ b/packages/server/src/providers/codex/inject.ts @@ -1,8 +1,15 @@ import { execa } from "execa"; +import { estimateTokenBaseline, formatBaselineSuffix } from "@threadle/shared"; import type { ContextPayload, InjectResult, InjectTarget } from "@threadle/shared"; -import { forEachLine, type LogLane, type LogSink } from "../stream.js"; +import { forEachLine, toolSummary, type LogLane, type LogSink } from "../stream.js"; import { positionalSafe } from "../argv-safe.js"; import { codexBin } from "./paths.js"; +import { + findCodexUsageFromRollout, + findCodexUsageInJsonl, + parseCodexUsageFromEvent, + type CodexUsage, +} from "./usage.js"; const INJECT_TIMEOUT_MS = 10 * 60_000; @@ -23,12 +30,16 @@ export function safetyArgs(opts?: { sandbox?: string; askForApproval?: string; }): string[] { - return [ - "--sandbox", - opts?.sandbox?.trim() || "workspace-write", - "--ask-for-approval", - opts?.askForApproval?.trim() || "never", - ]; + const sandbox = opts?.sandbox?.trim() || "workspace-write"; + // `--approve-for-me` implies workspace-write and cannot combine with `--sandbox`. + if (sandbox === "workspace-write") { + const ask = opts?.askForApproval?.trim(); + if (!ask || ask === "never" || ask === "on-failure") { + return ["--approve-for-me"]; + } + return ["--sandbox", "workspace-write"]; + } + return ["--sandbox", sandbox]; } export function execArgs(opts: { @@ -37,6 +48,8 @@ export function execArgs(opts: { prompt: string; sandbox?: string; askForApproval?: string; + /** Empty / non-git cwd — allow Codex outside a repo. */ + ignoreLocalMarkdown?: boolean; }): string[] { // codex exec [--json] [resume ] [flags…] const args = [ @@ -44,6 +57,9 @@ export function execArgs(opts: { "--json", ...safetyArgs({ sandbox: opts.sandbox, askForApproval: opts.askForApproval }), ]; + if (opts.ignoreLocalMarkdown) { + args.push("--skip-git-repo-check"); + } if (opts.model) args.push("--model", opts.model); if (opts.sessionId) args.push("resume", opts.sessionId); args.push(positionalSafe(opts.prompt)); @@ -93,7 +109,8 @@ function findResultText(stdout: string): string | undefined { return joined || undefined; } -function eventToLogs(line: string): Array<[LogLane, string]> { +/** Map one Codex exec --json line into log lanes (thinking / tool / text). */ +export function eventToLogs(line: string): Array<[LogLane, string]> { const out: Array<[LogLane, string]> = []; try { const evt = JSON.parse(line) as Record; @@ -101,18 +118,123 @@ function eventToLogs(line: string): Array<[LogLane, string]> { if (evt.type === "agent_message" && typeof evt.message === "string") { out.push(["text", evt.message]); } + + // Rollout-shaped items on the exec --json stream + const payload = + evt.payload && typeof evt.payload === "object" && !Array.isArray(evt.payload) + ? (evt.payload as Record) + : undefined; + const itemType = + (typeof payload?.type === "string" && payload.type) || + (typeof evt.type === "string" && evt.type) || + ""; + + if (itemType === "reasoning" || evt.type === "reasoning") { + const bag = payload ?? evt; + const summary = + typeof bag.summary === "string" + ? bag.summary + : Array.isArray(bag.summary) + ? bag.summary + .map((b) => + b && typeof b === "object" && typeof (b as { text?: string }).text === "string" + ? (b as { text: string }).text + : "", + ) + .filter(Boolean) + .join("\n") + : ""; + const content = + typeof bag.content === "string" + ? bag.content + : Array.isArray(bag.content) + ? bag.content + .map((b) => + b && typeof b === "object" && typeof (b as { text?: string }).text === "string" + ? (b as { text: string }).text + : "", + ) + .filter(Boolean) + .join("\n") + : ""; + const think = (summary || content).trim(); + if (think) out.push(["thinking", think.length > 200 ? `${think.slice(0, 200)}…` : think]); + } + + if ( + itemType === "function_call" || + itemType === "custom_tool_call" || + itemType === "tool_call" || + evt.type === "function_call" || + evt.type === "tool_call" + ) { + const bag = payload ?? evt; + const name = typeof bag.name === "string" ? bag.name : "tool"; + let input: unknown = bag.arguments ?? bag.input; + if (typeof input === "string") { + try { + input = JSON.parse(input); + } catch { + /* keep string */ + } + } + out.push(["tool", toolSummary(name, input)]); + } + + // Nested item on some stream events + const item = evt.item; + if (item && typeof item === "object" && !Array.isArray(item)) { + const it = item as Record; + if (typeof it.text === "string" && it.text) out.push(["text", it.text]); + if (it.type === "function_call" || it.type === "tool_call") { + const name = typeof it.name === "string" ? it.name : "tool"; + out.push(["tool", toolSummary(name, it.arguments ?? it.input)]); + } + } } catch { // ignore } return out; } +function toInjectUsage( + u: CodexUsage | undefined, +): InjectResult["usage"] | undefined { + if (!u || u.source !== "cli") return undefined; + if (!u.tokensIn && !u.tokensOut) return undefined; + return { inputTokens: u.tokensIn, outputTokens: u.tokensOut }; +} + +function logTokMeta( + onLog: LogSink | undefined, + usage: InjectResult["usage"] | undefined, + prompt: string | undefined, + ignoreLocalMarkdown: boolean | undefined, +): void { + if (!onLog || !usage) return; + const inTok = usage.inputTokens ?? 0; + const outTok = usage.outputTokens ?? 0; + const parts = [`${inTok}→${outTok} tok`]; + if (prompt) { + const b = estimateTokenBaseline(inTok, prompt, { ignoreLocalMarkdown }); + const suf = b ? formatBaselineSuffix(b) : ""; + if (suf) parts.push(suf); + } + onLog("meta", `■ ${parts.join(" · ")}`); +} + async function runCodexStreaming( args: string[], cwd: string, onLog?: LogSink, signal?: AbortSignal, -): Promise<{ sessionId?: string; resultText?: string; isError: boolean }> { +): Promise<{ + sessionId?: string; + resultText?: string; + stderr?: string; + isError: boolean; + usage?: CodexUsage; +}> { const child = execa(codexBin(), args, { cwd, timeout: INJECT_TIMEOUT_MS, @@ -121,19 +243,45 @@ async function runCodexStreaming( reject: false, }); const chunks: string[] = []; + const errChunks: string[] = []; + let streamUsage: CodexUsage | undefined; forEachLine(child.stdout, (line) => { chunks.push(line); + try { + const evt = JSON.parse(line) as Record; + const u = parseCodexUsageFromEvent(evt); + if (u) streamUsage = u; + } catch { + /* not json */ + } if (onLog) for (const [lane, text] of eventToLogs(line)) onLog(lane, text); }); + forEachLine(child.stderr, (line) => { + errChunks.push(line); + if (onLog) onLog("raw", line); + }); const result = await child; const stdout = chunks.join("\n") || result.stdout || ""; + const stderr = errChunks.join("\n") || result.stderr || ""; + const sessionId = findSessionId(stdout); + let usage = streamUsage ?? findCodexUsageInJsonl(stdout); + if ((!usage || usage.source !== "cli") && sessionId) { + usage = (await findCodexUsageFromRollout(sessionId)) ?? usage; + } return { - sessionId: findSessionId(stdout), + sessionId, resultText: findResultText(stdout), + stderr: stderr.trim() || undefined, isError: result.exitCode !== 0 && result.exitCode !== undefined, + usage, }; } +function codexFail(prefix: string, r: { resultText?: string; stderr?: string }): Error { + const detail = [r.stderr, r.resultText].filter(Boolean).join("\n").trim(); + return new Error(detail ? `${prefix}: ${detail}` : prefix); +} + export async function injectCodex( target: InjectTarget, payload: ContextPayload, @@ -148,22 +296,24 @@ export async function injectCodex( execArgs({ prompt, model: target.model, sessionId: target.sessionId }), cwd, ); - if (r.isError) throw new Error("codex continue inject failed"); + if (r.isError) throw codexFail("codex continue inject failed", r); return { newSessionId: r.sessionId ?? target.sessionId, provider: "codex", resultText: r.resultText, + usage: toInjectUsage(r.usage), }; } if (target.mode === "new-session") { const r = await runCodexStreaming(execArgs({ prompt, model: target.model }), cwd); - if (r.isError) throw new Error("codex new-session inject failed"); + if (r.isError) throw codexFail("codex new-session inject failed", r); if (!r.sessionId) throw new Error("codex new-session returned no session id"); return { newSessionId: r.sessionId, provider: "codex", resultText: r.resultText, + usage: toInjectUsage(r.usage), }; } @@ -178,9 +328,13 @@ export async function runCodexAgent(opts: { sessionId?: string; sandbox?: string; askForApproval?: string; + ignoreLocalMarkdown?: boolean; onLog?: LogSink; signal?: AbortSignal; }): Promise { + if (opts.onLog && opts.ignoreLocalMarkdown) { + opts.onLog("meta", "ignore local md · bare cwd + --skip-git-repo-check"); + } const r = await runCodexStreaming( execArgs({ prompt: opts.prompt, @@ -188,18 +342,22 @@ export async function runCodexAgent(opts: { sessionId: opts.sessionId, sandbox: opts.sandbox, askForApproval: opts.askForApproval, + ignoreLocalMarkdown: opts.ignoreLocalMarkdown, }), opts.projectDir, opts.onLog, opts.signal, ); - if (r.isError) throw new Error("codex agent run failed"); + if (r.isError) throw codexFail("codex agent run failed", r); const newSessionId = r.sessionId ?? opts.sessionId; if (!newSessionId) throw new Error("codex agent run returned no session id"); + const usage = toInjectUsage(r.usage); + logTokMeta(opts.onLog, usage, opts.prompt, opts.ignoreLocalMarkdown); return { newSessionId, provider: "codex", resultText: r.resultText, + usage, }; } diff --git a/packages/server/src/providers/codex/usage.ts b/packages/server/src/providers/codex/usage.ts index 8a7635e..091c309 100644 --- a/packages/server/src/providers/codex/usage.ts +++ b/packages/server/src/providers/codex/usage.ts @@ -1,11 +1,154 @@ +import fs from "node:fs"; +import path from "node:path"; import { readCodexTranscript } from "./jsonl.js"; +import { sessionsDir } from "./paths.js"; export interface CodexUsage { tokensIn: number; tokensOut: number; tokensReasoning?: number; - /** Public rollouts rarely carry authoritative usage — chars÷4 for now. */ - source: "estimate"; + /** CLI token_usage_record / token_count, or chars÷4 fallback. */ + source: "cli" | "estimate"; +} + +type TokenBag = { + input_tokens?: number; + output_tokens?: number; + reasoning_output_tokens?: number; +}; + +function fromBag( + bag: TokenBag | undefined, + source: "cli", +): CodexUsage | undefined { + if (!bag) return undefined; + const tokensIn = typeof bag.input_tokens === "number" ? bag.input_tokens : undefined; + const tokensOut = typeof bag.output_tokens === "number" ? bag.output_tokens : undefined; + if (tokensIn == null && tokensOut == null) return undefined; + const reasoning = + typeof bag.reasoning_output_tokens === "number" && bag.reasoning_output_tokens > 0 + ? bag.reasoning_output_tokens + : undefined; + return { + tokensIn: tokensIn ?? 0, + tokensOut: tokensOut ?? 0, + tokensReasoning: reasoning, + source, + }; +} + +/** + * Parse one Codex JSONL / `--json` event for authoritative usage. + * Prefers `token_usage_record`, then `token_count` last/total bags. + */ +export function parseCodexUsageFromEvent( + evt: Record, +): CodexUsage | undefined { + if (evt.type === "token_usage_record") { + const payload = + evt.payload && typeof evt.payload === "object" && !Array.isArray(evt.payload) + ? (evt.payload as Record) + : undefined; + const usage = payload?.usage; + if (usage && typeof usage === "object" && !Array.isArray(usage)) { + return fromBag(usage as TokenBag, "cli"); + } + } + + if (evt.type === "event_msg") { + const payload = + evt.payload && typeof evt.payload === "object" && !Array.isArray(evt.payload) + ? (evt.payload as Record) + : undefined; + if (payload?.type === "token_count") { + const info = + payload.info && typeof payload.info === "object" && !Array.isArray(payload.info) + ? (payload.info as Record) + : undefined; + return ( + fromBag(info?.last_token_usage as TokenBag | undefined, "cli") ?? + fromBag(info?.total_token_usage as TokenBag | undefined, "cli") + ); + } + } + + // Some exec streams nest usage on item / top-level usage bags. + if (evt.usage && typeof evt.usage === "object" && !Array.isArray(evt.usage)) { + const u = fromBag(evt.usage as TokenBag, "cli"); + if (u) return u; + } + + return undefined; +} + +/** Scan newline-delimited JSON (stdout or rollout) for the latest CLI usage. */ +export function findCodexUsageInJsonl(text: string): CodexUsage | undefined { + let last: CodexUsage | undefined; + for (const line of text.split("\n")) { + if (!line.trim()) continue; + try { + const evt = JSON.parse(line) as Record; + const u = parseCodexUsageFromEvent(evt); + if (u) last = u; + } catch { + // skip + } + } + return last; +} + +/** Walk recent session day dirs for a rollout matching `sessionId`. */ +export async function findCodexUsageFromRollout( + sessionId: string, +): Promise { + const root = sessionsDir(); + let years: string[]; + try { + years = await fs.promises.readdir(root); + } catch { + return undefined; + } + // Newest years/months/days first. + years.sort((a, b) => b.localeCompare(a)); + for (const y of years.slice(0, 2)) { + const yDir = path.join(root, y); + let months: string[]; + try { + months = await fs.promises.readdir(yDir); + } catch { + continue; + } + months.sort((a, b) => b.localeCompare(a)); + for (const m of months.slice(0, 3)) { + const mDir = path.join(yDir, m); + let days: string[]; + try { + days = await fs.promises.readdir(mDir); + } catch { + continue; + } + days.sort((a, b) => b.localeCompare(a)); + for (const d of days.slice(0, 7)) { + const dDir = path.join(mDir, d); + let files: string[]; + try { + files = await fs.promises.readdir(dDir); + } catch { + continue; + } + const hit = files.find((f) => f.includes(sessionId) && f.endsWith(".jsonl")); + if (!hit) continue; + try { + const raw = await fs.promises.readFile(path.join(dDir, hit), "utf8"); + const u = findCodexUsageInJsonl(raw); + if (u) return u; + } catch { + /* skip */ + } + } + } + } + return undefined; } /** @@ -62,5 +205,12 @@ export async function estimateUsageFromTranscript( export async function resolveCodexUsage( transcriptPath: string, ): Promise { + try { + const raw = await fs.promises.readFile(transcriptPath, "utf8"); + const cli = findCodexUsageInJsonl(raw); + if (cli) return cli; + } catch { + /* missing */ + } return estimateUsageFromTranscript(transcriptPath); } diff --git a/packages/server/src/providers/copilot/inject.ts b/packages/server/src/providers/copilot/inject.ts index 2d72a52..7936f51 100644 --- a/packages/server/src/providers/copilot/inject.ts +++ b/packages/server/src/providers/copilot/inject.ts @@ -1,8 +1,16 @@ import { randomUUID } from "node:crypto"; +import fs from "node:fs"; import { execa } from "execa"; +import { estimateTokenBaseline, formatBaselineSuffix } from "@threadle/shared"; import type { ContextPayload, InjectResult, InjectTarget } from "@threadle/shared"; import { forEachLine, type LogLane, type LogSink } from "../stream.js"; import { copilotBin } from "./paths.js"; +import { + copilotUsageTempPath, + readCopilotUsageFile, + waitSessionTokensFromDb, + type CopilotTokenUsage, +} from "./usage.js"; const INJECT_TIMEOUT_MS = 10 * 60_000; @@ -39,8 +47,13 @@ function execArgs(opts: { model?: string; agent?: string; prompt: string; + /** Skip AGENTS.md / custom instruction files. */ + ignoreLocalMarkdown?: boolean; + usageOutputFile?: string; }): string[] { const args = ["-p", opts.prompt, "-s", ...safetyArgs()]; + if (opts.ignoreLocalMarkdown) args.push("--no-custom-instructions"); + if (opts.usageOutputFile) args.push("--usage-output-file", opts.usageOutputFile); if (opts.model) args.push("--model", opts.model); if (opts.agent && opts.agent !== "copilot") args.push("--agent", opts.agent); if (opts.sessionId) { @@ -60,12 +73,36 @@ function findSessionIdFromArgs(args: string[]): string | undefined { return undefined; } +function logTokMeta( + onLog: LogSink | undefined, + usage: CopilotTokenUsage | undefined, + prompt: string | undefined, + ignoreLocalMarkdown: boolean | undefined, +): void { + if (!onLog || !usage) return; + const parts = [`${usage.inputTokens}→${usage.outputTokens} tok`]; + if (prompt) { + const b = estimateTokenBaseline(usage.inputTokens, prompt, { + ignoreLocalMarkdown, + }); + const suf = b ? formatBaselineSuffix(b) : ""; + if (suf) parts.push(suf); + } + onLog("meta", `■ ${parts.join(" · ")}`); +} + async function runCopilotStreaming( args: string[], cwd: string, onLog?: LogSink, signal?: AbortSignal, -): Promise<{ sessionId?: string; resultText?: string; isError: boolean }> { + usageFile?: string, +): Promise<{ + sessionId?: string; + resultText?: string; + isError: boolean; + usage?: CopilotTokenUsage; +}> { const child = execa(copilotBin(), args, { cwd, timeout: INJECT_TIMEOUT_MS, @@ -87,10 +124,19 @@ async function runCopilotStreaming( }); const result = await child; const stdout = chunks.join("\n") || result.stdout || ""; + const sessionId = findSessionIdFromArgs(args); + let usage = usageFile ? readCopilotUsageFile(usageFile) : undefined; + if (!usage && sessionId) { + usage = await waitSessionTokensFromDb(sessionId); + } + if (usageFile) { + await fs.promises.unlink(usageFile).catch(() => undefined); + } return { - sessionId: findSessionIdFromArgs(args), + sessionId, resultText: stdout.trim() || undefined, isError: result.exitCode !== 0 && result.exitCode !== undefined, + usage, }; } @@ -104,29 +150,34 @@ export async function injectCopilot( if (target.mode === "continue") { if (!target.sessionId) throw new Error("continue inject requires a sessionId"); + const usageFile = copilotUsageTempPath(); const args = execArgs({ prompt, model: target.model, sessionId: target.sessionId, + usageOutputFile: usageFile, }); - const r = await runCopilotStreaming(args, cwd); + const r = await runCopilotStreaming(args, cwd, undefined, undefined, usageFile); if (r.isError) throw new Error("copilot continue inject failed"); return { newSessionId: r.sessionId ?? target.sessionId, provider: "copilot", resultText: r.resultText, + usage: r.usage, }; } if (target.mode === "new-session") { - const args = execArgs({ prompt, model: target.model }); - const r = await runCopilotStreaming(args, cwd); + const usageFile = copilotUsageTempPath(); + const args = execArgs({ prompt, model: target.model, usageOutputFile: usageFile }); + const r = await runCopilotStreaming(args, cwd, undefined, undefined, usageFile); if (r.isError) throw new Error("copilot new-session inject failed"); if (!r.sessionId) throw new Error("copilot new-session returned no session id"); return { newSessionId: r.sessionId, provider: "copilot", resultText: r.resultText, + usage: r.usage, }; } @@ -139,23 +190,38 @@ export async function runCopilotAgent(opts: { prompt: string; projectDir: string; sessionId?: string; + ignoreLocalMarkdown?: boolean; onLog?: LogSink; signal?: AbortSignal; }): Promise { + const usageFile = copilotUsageTempPath(); const args = execArgs({ prompt: opts.prompt, model: opts.model, agent: opts.agent, sessionId: opts.sessionId, + ignoreLocalMarkdown: opts.ignoreLocalMarkdown, + usageOutputFile: usageFile, }); - const r = await runCopilotStreaming(args, opts.projectDir, opts.onLog, opts.signal); + if (opts.onLog && opts.ignoreLocalMarkdown) { + opts.onLog("meta", "ignore local md · --no-custom-instructions"); + } + const r = await runCopilotStreaming( + args, + opts.projectDir, + opts.onLog, + opts.signal, + usageFile, + ); if (r.isError) throw new Error("copilot agent run failed"); const newSessionId = r.sessionId ?? opts.sessionId; if (!newSessionId) throw new Error("copilot agent run returned no session id"); + logTokMeta(opts.onLog, r.usage, opts.prompt, opts.ignoreLocalMarkdown); return { newSessionId, provider: "copilot", resultText: r.resultText, + usage: r.usage, }; } diff --git a/packages/server/src/providers/copilot/usage.ts b/packages/server/src/providers/copilot/usage.ts index fc4e3db..b397a11 100644 --- a/packages/server/src/providers/copilot/usage.ts +++ b/packages/server/src/providers/copilot/usage.ts @@ -1,3 +1,7 @@ +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import { randomUUID } from "node:crypto"; import type { SessionRef } from "@threadle/shared"; import { sessionStoreDbPath } from "./paths.js"; @@ -12,6 +16,11 @@ interface UsageAgg { model?: string; } +export type CopilotTokenUsage = { + inputTokens: number; + outputTokens: number; +}; + function openDb(): SqliteDb | undefined { try { const sqlite = process.getBuiltinModule( @@ -23,6 +32,117 @@ function openDb(): SqliteDb | undefined { } } +function pickNum(obj: Record, ...keys: string[]): number | undefined { + for (const k of keys) { + const v = obj[k]; + if (typeof v === "number" && Number.isFinite(v)) return v; + } + return undefined; +} + +/** + * Parse Copilot `--usage-output-file` JSON (shape churns; accept common keys). + */ +export function parseCopilotUsageFile(raw: string): CopilotTokenUsage | undefined { + let data: unknown; + try { + data = JSON.parse(raw); + } catch { + return undefined; + } + const candidates: Record[] = []; + if (data && typeof data === "object" && !Array.isArray(data)) { + const root = data as Record; + candidates.push(root); + for (const key of ["usage", "totals", "session", "statistics", "stats"]) { + const nested = root[key]; + if (nested && typeof nested === "object" && !Array.isArray(nested)) { + candidates.push(nested as Record); + } + } + } + for (const c of candidates) { + const inputTokens = pickNum( + c, + "input_tokens", + "inputTokens", + "prompt_tokens", + "promptTokens", + ); + const outputTokens = pickNum( + c, + "output_tokens", + "outputTokens", + "completion_tokens", + "completionTokens", + ); + if (inputTokens != null || outputTokens != null) { + if (!(inputTokens || outputTokens)) continue; + return { inputTokens: inputTokens ?? 0, outputTokens: outputTokens ?? 0 }; + } + } + return undefined; +} + +/** Sum token columns for a session id from assistant_usage_events. */ +export function sessionTokensFromDb(sessionId: string): CopilotTokenUsage | undefined { + const db = openDb(); + if (!db) return undefined; + try { + const row = db + .prepare( + `SELECT + COALESCE(SUM(input_tokens), 0) AS tokensIn, + COALESCE(SUM(output_tokens), 0) AS tokensOut + FROM assistant_usage_events WHERE session_id = ?`, + ) + .get(sessionId) as { tokensIn?: number; tokensOut?: number } | undefined; + if (!row) return undefined; + const inputTokens = Number(row.tokensIn) || 0; + const outputTokens = Number(row.tokensOut) || 0; + if (!inputTokens && !outputTokens) return undefined; + return { inputTokens, outputTokens }; + } catch { + return undefined; + } finally { + try { + db.close(); + } catch { + /* ignore */ + } + } +} + +/** Brief settle + DB lookup for tokens written after a CLI exit. */ +export async function waitSessionTokensFromDb( + sessionId: string, + attempts = 4, + delayMs = 150, +): Promise { + for (let i = 0; i < attempts; i++) { + const u = sessionTokensFromDb(sessionId); + if (u) return u; + if (i + 1 < attempts) { + await new Promise((r) => setTimeout(r, delayMs * (i + 1))); + } + } + return undefined; +} + +/** Create a temp path for `--usage-output-file`. */ +export function copilotUsageTempPath(): string { + return path.join(os.tmpdir(), `threadle-copilot-usage-${randomUUID()}.json`); +} + +export function readCopilotUsageFile(filePath: string): CopilotTokenUsage | undefined { + try { + const raw = fs.readFileSync(filePath, "utf8"); + return parseCopilotUsageFile(raw); + } catch { + return undefined; + } +} + /** Sum token columns from assistant_usage_events onto a SessionRef. */ export function enrichSessionTokens(ref: SessionRef): SessionRef { const db = openDb(); diff --git a/packages/server/src/providers/cursor/inject.ts b/packages/server/src/providers/cursor/inject.ts index dc1465b..af1d941 100644 --- a/packages/server/src/providers/cursor/inject.ts +++ b/packages/server/src/providers/cursor/inject.ts @@ -1,6 +1,11 @@ import { execa } from "execa"; import type { ContextPayload, InjectResult, InjectTarget } from "@threadle/shared"; import { enrichUsageExhaustionError } from "@threadle/shared"; +import { formatDuration } from "../run-metrics.js"; +import { + estimateTokenBaseline, + formatBaselineSuffix, +} from "@threadle/shared"; import { forEachLine, toolSummary, type LogLane, type LogSink } from "../stream.js"; import { positionalSafe } from "../argv-safe.js"; import { agentBin } from "./agent-bin.js"; @@ -26,6 +31,7 @@ interface CursorJsonResult { is_error?: boolean; result?: string; session_id?: string; + duration_ms?: number; usage?: { inputTokens?: number; outputTokens?: number; @@ -186,11 +192,35 @@ function cursorEventToLogs(line: string): Array<[LogLane, string]> { return out; } +function cursorResultMeta( + evt: CursorJsonResult, + opts?: { prompt?: string; ignoreLocalMarkdown?: boolean }, +): string { + const parts: string[] = []; + if (typeof evt.duration_ms === "number") parts.push(formatDuration(evt.duration_ms)); + const u = evt.usage; + if (u) { + const inTok = u.inputTokens ?? 0; + const outTok = u.outputTokens ?? 0; + if (inTok || outTok) parts.push(`${inTok}→${outTok} tok`); + if (opts?.prompt && inTok) { + const b = estimateTokenBaseline(inTok, opts.prompt, { + ignoreLocalMarkdown: opts.ignoreLocalMarkdown, + }); + const suf = b ? formatBaselineSuffix(b) : ""; + if (suf) parts.push(suf); + } + } + if (evt.subtype) parts.push(evt.subtype); + return parts.length ? `■ ${parts.join(" · ")}` : "■ done"; +} + async function runAgentStreaming( args: string[], cwd: string, onLog: LogSink, signal?: AbortSignal, + metaOpts?: { prompt?: string; ignoreLocalMarkdown?: boolean }, ): Promise<{ sessionId?: string; resultText?: string; @@ -199,7 +229,7 @@ async function runAgentStreaming( usage?: CursorJsonResult["usage"]; }> { const child = execa( - "agent", + agentBin(), [...args, "--output-format", "stream-json"], { cwd, @@ -216,7 +246,10 @@ async function runAgentStreaming( try { const evt = JSON.parse(line) as CursorJsonResult & { session_id?: string }; if (typeof evt.session_id === "string") sessionId = evt.session_id; - if (evt.type === "result") result = evt; + if (evt.type === "result") { + result = evt; + onLog("meta", cursorResultMeta(evt, metaOpts)); + } for (const [lane, text] of cursorEventToLogs(line)) onLog(lane, text); } catch { // ignore @@ -325,6 +358,8 @@ export interface RunCursorAgentOptions { prompt: string; projectDir: string; sessionId?: string; + /** Caller should pass an empty bare workspace (no AGENTS.md / .cursor/rules). */ + ignoreLocalMarkdown?: boolean; onLog?: LogSink; signal?: AbortSignal; } @@ -332,6 +367,9 @@ export interface RunCursorAgentOptions { export async function runCursorAgent( opts: RunCursorAgentOptions, ): Promise { + if (opts.onLog && opts.ignoreLocalMarkdown) { + opts.onLog("meta", "ignore local md · bare workspace (no AGENTS.md / rules)"); + } const chatId = opts.sessionId ?? (await createChatId()); const args = baseArgs({ sessionId: chatId, @@ -342,7 +380,10 @@ export async function runCursorAgent( applyAgentMode(args, opts.agent); const result = opts.onLog - ? await runAgentStreaming(args, opts.projectDir, opts.onLog, opts.signal) + ? await runAgentStreaming(args, opts.projectDir, opts.onLog, opts.signal, { + prompt: opts.prompt, + ignoreLocalMarkdown: opts.ignoreLocalMarkdown, + }) : await runAgentJson(args, opts.projectDir); if (result.isError) { @@ -357,6 +398,12 @@ export async function runCursorAgent( newSessionId, provider: "cursor", resultText: result.resultText, + usage: result.usage + ? { + inputTokens: result.usage.inputTokens, + outputTokens: result.usage.outputTokens, + } + : undefined, }; } diff --git a/packages/server/src/providers/freshness/fixtures.ts b/packages/server/src/providers/freshness/fixtures.ts index 4605cef..3d166ef 100644 --- a/packages/server/src/providers/freshness/fixtures.ts +++ b/packages/server/src/providers/freshness/fixtures.ts @@ -17,6 +17,10 @@ import { _resetDbForTests } from "../opencode/db.js"; import { readOpencodeTranscript } from "../opencode/normalize.js"; import { discoverSessions } from "../grok/discover.js"; import { readGrokTranscript } from "../grok/transcript.js"; +import { + discoverSessions as discoverMuseSessions, +} from "../muse/discover.js"; +import { readMuseTranscript } from "../muse/transcript.js"; export interface FixtureProbeResult { id: string; @@ -251,6 +255,42 @@ async function probeGrok(root: string): Promise { } } +async function probeMuse(root: string): Promise { + const museShare = fixture(root, "muse"); + const sessionId = "01a0ce3e-daea-76f0-99fc-d53076374955"; + const prev = process.env.MUSE_DATA_DIR; + try { + process.env.MUSE_DATA_DIR = museShare; + const refs = await discoverMuseSessions(); + if (refs.length < 1) { + return { + id: "fixture:muse", + ok: false, + skipped: false, + detail: `${museShare} — expected ≥1 session, got ${refs.length}`, + }; + } + const msgs = await readMuseTranscript(sessionId); + if (msgs.length < 2) { + return { + id: "fixture:muse", + ok: false, + skipped: false, + detail: `${museShare} — expected ≥2 transcript msgs, got ${msgs.length}`, + }; + } + return { + id: "fixture:muse", + ok: true, + skipped: false, + detail: `${museShare} · ${refs.length} sessions · ${msgs.length} msgs`, + }; + } finally { + if (prev === undefined) delete process.env.MUSE_DATA_DIR; + else process.env.MUSE_DATA_DIR = prev; + } +} + /** * Parse every golden provider fixture. Missing fixture tree → all skipped. * Failures (parse/shape) set ok=false (not skipped). @@ -275,6 +315,7 @@ export async function probeGoldenFixtures(): Promise { () => probeAntigravity(root), () => probeOpencode(root), () => probeGrok(root), + () => probeMuse(root), ]; const out: FixtureProbeResult[] = []; diff --git a/packages/server/src/providers/freshness/inject-flags.ts b/packages/server/src/providers/freshness/inject-flags.ts index fb4154d..6dd5021 100644 --- a/packages/server/src/providers/freshness/inject-flags.ts +++ b/packages/server/src/providers/freshness/inject-flags.ts @@ -7,7 +7,7 @@ export interface InjectFlagProbe { id: string; /** How to resolve the binary (PATH name or absolute). */ - bin: "claude" | "opencode" | "cursor-agent" | "agy" | "codex" | "copilot" | "grok"; + bin: "claude" | "opencode" | "cursor-agent" | "agy" | "codex" | "copilot" | "grok" | "muse"; /** * Substrings that count as present (any match). Use several when help uses * bracket notation (`--foo[-bar]`) or aliases. @@ -98,10 +98,10 @@ export const INJECT_FLAG_PROBES: InjectFlagProbe[] = [ why: "workspace-write inject", }, { - id: "codex:ask-for-approval", + id: "codex:approve-for-me", bin: "codex", - match: ["--ask-for-approval"], - why: "never for headless", + match: ["--approve-for-me"], + why: "headless auto-approval (replaced --ask-for-approval)", }, { id: "copilot:prompt", @@ -145,4 +145,38 @@ export const INJECT_FLAG_PROBES: InjectFlagProbe[] = [ match: ["--session-id", "-s"], why: "pin new session UUID", }, + { + id: "muse:exec", + bin: "muse", + match: ["exec"], + why: "muse exec headless inject", + }, + { + id: "muse:json", + bin: "muse", + match: ["--json"], + why: "jsonl event stream", + helpArgs: ["exec", "--help"], + }, + { + id: "muse:prompt-file", + bin: "muse", + match: ["--prompt-file"], + why: "context file inject", + helpArgs: ["exec", "--help"], + }, + { + id: "muse:approval-mode", + bin: "muse", + match: ["--approval-mode"], + why: "non-interactive approval never", + helpArgs: ["exec", "--help"], + }, + { + id: "muse:session-id", + bin: "muse", + match: ["--session-id"], + why: "continue / pin session UUID", + helpArgs: ["exec", "--help"], + }, ]; diff --git a/packages/server/src/providers/freshness/probe.ts b/packages/server/src/providers/freshness/probe.ts index 7346fef..2d817e1 100644 --- a/packages/server/src/providers/freshness/probe.ts +++ b/packages/server/src/providers/freshness/probe.ts @@ -5,6 +5,7 @@ import { agyBin } from "../antigravity/agy-bin.js"; import { codexBin } from "../codex/paths.js"; import { copilotBin } from "../copilot/paths.js"; import { grokBin } from "../grok/paths.js"; +import { museBin } from "../muse/paths.js"; import { INJECT_FLAG_PROBES, type InjectFlagProbe } from "./inject-flags.js"; export interface FlagProbeResult { @@ -81,6 +82,15 @@ async function resolveBin(kind: InjectFlagProbe["bin"]): Promise { +): Promise<{ sessionId?: string; resultText?: string; stderr?: string; isError: boolean }> { const child = execa(grokBin(), args, { cwd, timeout: INJECT_TIMEOUT_MS, @@ -63,15 +87,20 @@ async function runGrokStreaming( reject: false, }); const chunks: string[] = []; + const errChunks: string[] = []; forEachLine(child.stdout, (line) => { chunks.push(line); if (onLog) onLog("text" as LogLane, line); }); forEachLine(child.stderr, (line) => { + errChunks.push(line); if (onLog) onLog("raw" as LogLane, line); }); const result = await child; const stdout = chunks.join("\n") || result.stdout || ""; + const stderr = errChunks.join("\n") || result.stderr || ""; + const combined = `${stdout}\n${stderr}`; + const paidFail = /402|payment required|balance exhausted|insufficient/i.test(combined); let sessionId: string | undefined; const resumeIdx = args.indexOf("--resume"); @@ -82,10 +111,17 @@ async function runGrokStreaming( return { sessionId, resultText: stdout.trim() || undefined, - isError: result.exitCode !== 0 && result.exitCode !== undefined, + stderr: stderr.trim() || undefined, + isError: + (result.exitCode !== 0 && result.exitCode !== undefined) || paidFail, }; } +function grokFail(prefix: string, r: { resultText?: string; stderr?: string }): Error { + const detail = [r.stderr, r.resultText].filter(Boolean).join("\n").trim(); + return new Error(detail ? `${prefix}: ${detail}` : prefix); +} + export async function injectGrok( target: InjectTarget, payload: ContextPayload, @@ -102,11 +138,14 @@ export async function injectGrok( sessionId: target.sessionId, }); const r = await runGrokStreaming(args, cwd); - if (r.isError) throw new Error("grok continue inject failed"); + if (r.isError) throw grokFail("grok continue inject failed", r); + const id = r.sessionId ?? target.sessionId; + const usage = await waitGrokUsage(id, cwd); return { - newSessionId: r.sessionId ?? target.sessionId, + newSessionId: id, provider: "grok", resultText: r.resultText, + usage, }; } @@ -114,11 +153,14 @@ export async function injectGrok( const newSessionId = randomUUID(); const args = execArgs({ prompt, model: target.model, newSessionId }); const r = await runGrokStreaming(args, cwd); - if (r.isError) throw new Error("grok new-session inject failed"); + if (r.isError) throw grokFail("grok new-session inject failed", r); + const id = r.sessionId ?? newSessionId; + const usage = await waitGrokUsage(id, cwd); return { - newSessionId: r.sessionId ?? newSessionId, + newSessionId: id, provider: "grok", resultText: r.resultText, + usage, }; } @@ -133,23 +175,36 @@ export async function runGrokAgent(opts: { sessionId?: string; onLog?: LogSink; signal?: AbortSignal; + harnessExtras?: boolean; + ignoreLocalMarkdown?: boolean; }): Promise { const newSessionId = opts.sessionId ? undefined : randomUUID(); + const harnessExtras = await resolveRunHarnessExtras(opts.harnessExtras); const args = execArgs({ prompt: opts.prompt, model: opts.model, agent: opts.agent, sessionId: opts.sessionId, newSessionId, + harnessExtras, }); + if (opts.onLog) { + logHarnessExtrasLanes(opts.onLog, "grok", harnessExtras); + if (opts.ignoreLocalMarkdown) { + opts.onLog("meta", "ignore local md · bare cwd only (no skip flag)"); + } + } const r = await runGrokStreaming(args, opts.projectDir, opts.onLog, opts.signal); - if (r.isError) throw new Error("grok agent run failed"); + if (r.isError) throw grokFail("grok agent run failed", r); const id = r.sessionId ?? opts.sessionId ?? newSessionId; if (!id) throw new Error("grok agent run returned no session id"); + const usage = await waitGrokUsage(id, opts.projectDir); + logTokMeta(opts.onLog, usage, opts.prompt, opts.ignoreLocalMarkdown); return { newSessionId: id, provider: "grok", resultText: r.resultText, + usage, }; } diff --git a/packages/server/src/providers/grok/usage.ts b/packages/server/src/providers/grok/usage.ts index e2d59c8..c9cd738 100644 --- a/packages/server/src/providers/grok/usage.ts +++ b/packages/server/src/providers/grok/usage.ts @@ -1,6 +1,7 @@ import fs from "node:fs"; import path from "node:path"; import type { SessionRef } from "@threadle/shared"; +import { sessionsRoot } from "./paths.js"; /** Docs: costUsdTicks is 10¹⁰ ticks per USD. */ const TICKS_PER_USD = 1e10; @@ -16,6 +17,74 @@ interface UsageFile { }; } +export type GrokTokenUsage = { + inputTokens: number; + outputTokens: number; +}; + +/** Read usage.json → inject-shaped tokens (nonzero only). */ +export function readGrokUsageFile(usagePath: string): GrokTokenUsage | undefined { + try { + const raw = fs.readFileSync(usagePath, "utf8"); + const data = JSON.parse(raw) as UsageFile; + const s = data.session; + if (!s) return undefined; + const inputTokens = typeof s.inputTokens === "number" ? s.inputTokens : 0; + const outputTokens = typeof s.outputTokens === "number" ? s.outputTokens : 0; + if (!inputTokens && !outputTokens) return undefined; + return { inputTokens, outputTokens }; + } catch { + return undefined; + } +} + +/** Locate `…/sessions///usage.json` under GROK_HOME. */ +export function findGrokUsagePath( + sessionId: string, + projectDir?: string, +): string | undefined { + const root = sessionsRoot(); + if (projectDir) { + const direct = path.join( + root, + encodeURIComponent(projectDir), + sessionId, + "usage.json", + ); + if (fs.existsSync(direct)) return direct; + } + try { + for (const g of fs.readdirSync(root, { withFileTypes: true })) { + if (!g.isDirectory() || g.name.startsWith(".")) continue; + const candidate = path.join(root, g.name, sessionId, "usage.json"); + if (fs.existsSync(candidate)) return candidate; + } + } catch { + /* missing root */ + } + return undefined; +} + +/** Brief settle for usage.json after a CLI exit (often written late / starts at 0). */ +export async function waitGrokUsage( + sessionId: string, + projectDir?: string, + attempts = 5, + delayMs = 200, +): Promise { + for (let i = 0; i < attempts; i++) { + const p = findGrokUsagePath(sessionId, projectDir); + if (p) { + const u = readGrokUsageFile(p); + if (u) return u; + } + if (i + 1 < attempts) { + await new Promise((r) => setTimeout(r, delayMs * (i + 1))); + } + } + return undefined; +} + export function enrichSessionTokens( ref: SessionRef, sessionDir: string, diff --git a/packages/server/src/providers/harness-extras.ts b/packages/server/src/providers/harness-extras.ts new file mode 100644 index 0000000..ca4dd30 --- /dev/null +++ b/packages/server/src/providers/harness-extras.ts @@ -0,0 +1,43 @@ +import type { LogSink } from "./stream.js"; +import { PROVIDER_HARNESS_EXTRAS, type ProviderId } from "@threadle/shared"; + +/** + * Resolve whether post-answer harness extras should run for a provider spawn. + * Node/API override wins; otherwise Settings `harnessExtras` (legacy `museReminders`). + * Default false. + */ +export async function resolveRunHarnessExtras(override?: boolean): Promise { + if (override !== undefined) return override; + try { + const { readSettings } = await import("../routes/settings.js"); + const s = await readSettings(); + return s.harnessExtras === true || s.museReminders === true; + } catch { + return false; + } +} + +/** + * Emit one meta header + a line per harness lane (run / skipped) into the run log. + */ +export function logHarnessExtrasLanes( + onLog: LogSink | undefined, + provider: ProviderId | string, + enabled: boolean, +): void { + if (!onLog) return; + const spec = PROVIDER_HARNESS_EXTRAS[provider as ProviderId]; + if (!spec?.lanes.length) { + onLog("meta", enabled ? "extras on" : "extras off"); + return; + } + onLog( + "meta", + enabled + ? `extras on · ${spec.lanes.length} lane${spec.lanes.length === 1 ? "" : "s"}` + : `extras off · ${spec.lanes.length} lane${spec.lanes.length === 1 ? "" : "s"} skipped`, + ); + for (const name of spec.lanes) { + onLog("meta", enabled ? `· ${name} · run` : `⊘ ${name} · skipped`); + } +} diff --git a/packages/server/src/providers/muse/agents.ts b/packages/server/src/providers/muse/agents.ts new file mode 100644 index 0000000..0159dfa --- /dev/null +++ b/packages/server/src/providers/muse/agents.ts @@ -0,0 +1,86 @@ +import fs from "node:fs"; +import path from "node:path"; +import type { AgentDef } from "@threadle/shared"; +import { skillsRoot } from "./paths.js"; + +const BUILTIN: Array> = [ + { + name: "muse", + description: "Muse Code — Meta Muse Spark coding agent (default)", + kind: "primary", + }, +]; + +async function scanSkillMd(dir: string, scope: AgentDef["scope"]): Promise { + const out: AgentDef[] = []; + async function walk(d: string): Promise { + let entries: fs.Dirent[]; + try { + entries = await fs.promises.readdir(d, { withFileTypes: true }); + } catch { + return; + } + const skillFile = entries.find( + (e) => e.isFile() && /^skill\.md$/i.test(e.name), + ); + if (skillFile) { + const src = path.join(d, skillFile.name); + let name = path.basename(d); + let description: string | undefined; + let raw: string | undefined; + try { + raw = await fs.promises.readFile(src, "utf8"); + const fm = /^---\s*\n([\s\S]*?)\n---/.exec(raw); + if (fm) { + const nameM = /^name:\s*["']?(.+?)["']?\s*$/m.exec(fm[1]!); + const descM = /^description:\s*["']?(.+?)["']?\s*$/m.exec(fm[1]!); + if (nameM) name = nameM[1]!.trim(); + if (descM) description = descM[1]!.trim(); + } + } catch { + /* */ + } + out.push({ + provider: "muse", + name, + description, + source: src, + scope, + kind: "skill", + raw, + }); + return; + } + for (const ent of entries) { + if (ent.isDirectory() && !ent.name.startsWith(".")) { + await walk(path.join(d, ent.name)); + } + } + } + await walk(dir); + return out; +} + +export async function listMuseAgents(): Promise { + const defs: AgentDef[] = BUILTIN.map((b) => ({ + ...b, + provider: "muse" as const, + source: "muse:builtin", + scope: "builtin" as const, + })); + const root = skillsRoot(); + for (const [sub, scope] of [ + ["bundled", "builtin"], + ["user", "user"], + ["project", "project"], + ] as const) { + const dir = path.join(root, sub); + try { + await fs.promises.access(dir); + defs.push(...(await scanSkillMd(dir, scope))); + } catch { + /* missing */ + } + } + return defs; +} diff --git a/packages/server/src/providers/muse/discover.ts b/packages/server/src/providers/muse/discover.ts new file mode 100644 index 0000000..31d9c43 --- /dev/null +++ b/packages/server/src/providers/muse/discover.ts @@ -0,0 +1,352 @@ +import fs from "node:fs"; +import path from "node:path"; +import type { SessionRef, SessionStatus, TouchedFile } from "@threadle/shared"; +import { runtimeSessionsDir, sessionsRoot, sessionsRootAvailable } from "./paths.js"; + +type Json = Record; + +function microsToMs(us: unknown): number { + if (typeof us !== "number" || !Number.isFinite(us)) return 0; + // Muse timestamps are microseconds since epoch + return us > 1e15 ? Math.floor(us / 1000) : Math.floor(us); +} + +function asObj(v: unknown): Json | undefined { + return v && typeof v === "object" && !Array.isArray(v) ? (v as Json) : undefined; +} + +function* recordsFromLine(line: string): Generator { + let outer: Json; + try { + outer = JSON.parse(line) as Json; + } catch { + return; + } + yield outer; + const children = outer.children; + if (!Array.isArray(children)) return; + for (const ch of children) { + const raw = asObj(ch)?.record_json; + if (typeof raw !== "string") continue; + try { + yield JSON.parse(raw) as Json; + } catch { + /* skip */ + } + } +} + +export interface MuseSessionMeta { + id: string; + sessionDir: string; + logPath: string; + projectDir: string; + title?: string; + model?: string; + parentId?: string; + createdAt: number; + updatedAt: number; + messageCount: number; + tokensIn?: number; + tokensOut?: number; + tokensCacheRead?: number; + tokensReasoning?: number; + kind: "session" | "subagent-run"; +} + +async function scanLog(logPath: string): Promise & { firstPrompt?: string }> { + const out: Partial & { firstPrompt?: string } = {}; + let raw: string; + try { + raw = await fs.promises.readFile(logPath, "utf8"); + } catch { + return out; + } + let msgCount = 0; + for (const line of raw.split("\n")) { + if (!line.trim()) continue; + for (const rj of recordsFromLine(line)) { + const pt = typeof rj.payload_type === "string" ? rj.payload_type : ""; + const ts = microsToMs(rj.recorded_at); + if (ts) { + if (!out.createdAt || ts < out.createdAt) out.createdAt = ts; + if (!out.updatedAt || ts > out.updatedAt) out.updatedAt = ts; + } + const payload = asObj(rj.payload) ?? {}; + const record = asObj(payload.record) ?? payload; + + if (pt === "runtime.session.metadata") { + const root = record.workspace_root; + if (typeof root === "string" && root.trim()) out.projectDir = root.trim(); + } + if (pt === "runtime.session.route_facts") { + const cwd = record.cwd; + if (typeof cwd === "string" && cwd.trim() && !out.projectDir) { + out.projectDir = cwd.trim(); + } + } + if (pt === "session.name.changed") { + const name = payload.new_name; + if (typeof name === "string" && name.trim()) out.title = name.trim(); + } + if (pt === "run.model.configured") { + const mid = record.model_id ?? record.display_label; + if (typeof mid === "string" && mid.trim()) out.model = mid.trim(); + } + if (pt === "runtime.user_intent.accepted") { + msgCount += 1; + if (!out.firstPrompt) { + const blocks = payload.refill_blocks; + if (Array.isArray(blocks)) { + for (const b of blocks) { + const t = asObj(b)?.text; + if (typeof t === "string" && t.trim()) { + out.firstPrompt = t.trim(); + break; + } + } + } + } + } + if (pt === "runtime.session") { + const event = asObj(payload.event); + if (payload.kind === "run" && event?.kind === "assistant_message_committed") { + msgCount += 1; + } + if (payload.kind === "run" && event?.kind === "model_completed") { + const usage = asObj(event.usage); + if (usage) { + const tin = usage.input_tokens; + const tout = usage.output_tokens; + const cache = usage.cache_read_tokens ?? usage.cached_tokens; + const reason = usage.reasoning_tokens; + if (typeof tin === "number") out.tokensIn = (out.tokensIn ?? 0) + tin; + if (typeof tout === "number") out.tokensOut = (out.tokensOut ?? 0) + tout; + if (typeof cache === "number") { + out.tokensCacheRead = (out.tokensCacheRead ?? 0) + cache; + } + if (typeof reason === "number") { + out.tokensReasoning = (out.tokensReasoning ?? 0) + reason; + } + if (typeof event.model === "string" && event.model.trim()) { + out.model = event.model.trim(); + } + } + } + } + } + } + out.messageCount = msgCount; + return out; +} + +function toRef(meta: BuildMeta): SessionRef { + const title = + meta.title || + (meta.firstPrompt ? meta.firstPrompt.slice(0, 80) : undefined) || + `muse ${meta.id.slice(0, 8)}`; + const ref: SessionRef = { + provider: "muse", + id: meta.id, + projectDir: meta.projectDir || "", + title, + model: meta.model, + createdAt: meta.createdAt || meta.updatedAt, + updatedAt: meta.updatedAt || meta.createdAt || Date.now(), + status: "idle", + kind: meta.kind, + messageCount: meta.messageCount, + tokensIn: meta.tokensIn, + tokensOut: meta.tokensOut, + tokensCacheRead: meta.tokensCacheRead, + tokensReasoning: meta.tokensReasoning, + actualCost: 0, + meta: { + sessionDir: meta.sessionDir, + logPath: meta.logPath, + ...(meta.tokensIn !== undefined || meta.tokensOut !== undefined + ? { tokenSource: "transcript" } + : {}), + }, + }; + if (meta.parentId) ref.parentId = meta.parentId; + return ref; +} + +type BuildMeta = MuseSessionMeta & { firstPrompt?: string }; + +const sessionCache = new Map(); + +async function collectSessionDirs(root: string): Promise< + Array<{ id: string; sessionDir: string; logPath: string; parentId?: string }> +> { + const out: Array<{ id: string; sessionDir: string; logPath: string; parentId?: string }> = []; + async function walk(dir: string, parentId?: string): Promise { + let entries: fs.Dirent[]; + try { + entries = await fs.promises.readdir(dir, { withFileTypes: true }); + } catch { + return; + } + const logPath = path.join(dir, "session.jsonl"); + try { + await fs.promises.access(logPath, fs.constants.R_OK); + const base = path.basename(dir); + if (/^[0-9a-f-]{16,}$/i.test(base)) { + out.push({ id: base, sessionDir: dir, logPath, parentId }); + } + } catch { + /* not a session dir */ + } + for (const ent of entries) { + if (!ent.isDirectory()) continue; + if (ent.name.startsWith(".")) continue; + const child = path.join(dir, ent.name); + if (ent.name === "subagent") { + let kids: fs.Dirent[]; + try { + kids = await fs.promises.readdir(child, { withFileTypes: true }); + } catch { + continue; + } + // parent id from enclosing session dir + const enclosing = path.basename(dir); + const pid = /^[0-9a-f-]{16,}$/i.test(enclosing) ? enclosing : parentId; + for (const k of kids) { + if (k.isDirectory()) await walk(path.join(child, k.name), pid); + } + } else { + await walk(child, parentId); + } + } + } + await walk(root); + return out; +} + +async function loadMeta( + entry: { id: string; sessionDir: string; logPath: string; parentId?: string }, +): Promise { + const scanned = await scanLog(entry.logPath); + let st: fs.Stats | undefined; + try { + st = await fs.promises.stat(entry.logPath); + } catch { + /* */ + } + const updatedAt = + scanned.updatedAt || st?.mtimeMs || Date.now(); + const createdAt = scanned.createdAt || st?.birthtimeMs || updatedAt; + const meta: BuildMeta = { + id: entry.id, + sessionDir: entry.sessionDir, + logPath: entry.logPath, + projectDir: scanned.projectDir || "", + title: scanned.title, + model: scanned.model, + parentId: entry.parentId, + createdAt, + updatedAt, + messageCount: scanned.messageCount ?? 0, + tokensIn: scanned.tokensIn, + tokensOut: scanned.tokensOut, + tokensCacheRead: scanned.tokensCacheRead, + tokensReasoning: scanned.tokensReasoning, + kind: entry.parentId ? "subagent-run" : "session", + firstPrompt: scanned.firstPrompt, + }; + sessionCache.set(entry.id, meta); + return meta; +} + +export async function discoverSessions(): Promise { + if (!sessionsRootAvailable()) return []; + sessionCache.clear(); + const entries = await collectSessionDirs(sessionsRoot()); + // Top-level only for listSessions (exclude children — they come via listChildren) + const tops = entries.filter((e) => !e.parentId); + const refs: SessionRef[] = []; + for (const e of tops) { + refs.push(toRef(await loadMeta(e))); + } + // Still cache children for listChildren / getSession + for (const e of entries.filter((x) => x.parentId)) { + await loadMeta(e); + } + refs.sort((a, b) => b.updatedAt - a.updatedAt); + return refs; +} + +export async function childrenOf(sessionId: string): Promise { + if (!sessionsRootAvailable()) return []; + if (sessionCache.size === 0) await discoverSessions(); + const kids = [...sessionCache.values()].filter((m) => m.parentId === sessionId); + return kids.map(toRef).sort((a, b) => b.updatedAt - a.updatedAt); +} + +export async function getSessionRow(sessionId: string): Promise { + if (sessionCache.has(sessionId)) return toRef(sessionCache.get(sessionId)!); + if (!sessionsRootAvailable()) return undefined; + const entries = await collectSessionDirs(sessionsRoot()); + const hit = entries.find((e) => e.id === sessionId); + if (!hit) return undefined; + return toRef(await loadMeta(hit)); +} + +export async function resolveSessionDir(sessionId: string): Promise { + if (sessionCache.has(sessionId)) return sessionCache.get(sessionId)!.sessionDir; + await getSessionRow(sessionId); + return sessionCache.get(sessionId)?.sessionDir; +} + +export async function resolveLogPath(sessionId: string): Promise { + if (sessionCache.has(sessionId)) return sessionCache.get(sessionId)!.logPath; + await getSessionRow(sessionId); + return sessionCache.get(sessionId)?.logPath; +} + +export async function liveStatuses(): Promise> { + const map = new Map(); + let entries: fs.Dirent[]; + try { + entries = await fs.promises.readdir(runtimeSessionsDir(), { withFileTypes: true }); + } catch { + return map; + } + for (const ent of entries) { + if (!ent.isFile() || !ent.name.endsWith(".json")) continue; + try { + const raw = await fs.promises.readFile( + path.join(runtimeSessionsDir(), ent.name), + "utf8", + ); + const row = JSON.parse(raw) as Json; + const id = typeof row.session_id === "string" ? row.session_id : ent.name.replace(/\.json$/, ""); + const hint = typeof row.process_generation_hint === "string" ? row.process_generation_hint : ""; + const m = /pid=(\d+)/.exec(hint); + if (m) { + try { + process.kill(Number(m[1]), 0); + map.set(id, "live"); + continue; + } catch { + /* dead */ + } + } + if (row.target_eligibility === "message_capable") { + map.set(id, "live"); + } + } catch { + /* skip */ + } + } + return map; +} + +export async function listSessionFiles(sessionId: string): Promise { + // MSP tool file paths are not yet stably attributed; return empty rather than invent. + void sessionId; + return []; +} + +export { recordsFromLine, microsToMs, asObj }; diff --git a/packages/server/src/providers/muse/index.ts b/packages/server/src/providers/muse/index.ts new file mode 100644 index 0000000..931f446 --- /dev/null +++ b/packages/server/src/providers/muse/index.ts @@ -0,0 +1,121 @@ +import fs from "node:fs"; +import { execa } from "execa"; +import type { + AgentDef, + ContextPayload, + InjectResult, + InjectTarget, + NormalizedMessage, + SessionRef, + SessionStatus, + TouchedFile, +} from "@threadle/shared"; +import { sameProjectDir } from "../../paths.js"; +import type { + InjectOptions, + ListSessionsOptions, + SessionProvider, +} from "../types.js"; +import { listMuseAgents } from "./agents.js"; +import { + childrenOf, + discoverSessions, + getSessionRow, + listSessionFiles, + liveStatuses, +} from "./discover.js"; +import { injectMuse } from "./inject.js"; +import { museBin, museConfigHome, museShare, sessionsRootAvailable } from "./paths.js"; +import { readMuseTranscript } from "./transcript.js"; + +export class MuseProvider implements SessionProvider { + readonly id = "muse" as const; + #version: string | undefined; + + async available(): Promise { + if (sessionsRootAvailable()) return true; + try { + await fs.promises.access(museShare()); + return true; + } catch { + try { + await fs.promises.access(museConfigHome()); + return true; + } catch { + try { + await execa(museBin(), ["--version"], { timeout: 8_000 }); + return true; + } catch { + return false; + } + } + } + } + + async version(): Promise { + if (this.#version) return this.#version; + try { + const { stdout } = await execa(museBin(), ["--version"], { timeout: 10_000 }); + const line = stdout.trim().split("\n")[0]?.trim(); + // "Muse Code 1.3.0 (1.3.0-R3401.1)" + const m = /(\d+\.\d+\.\d+)/.exec(line ?? ""); + this.#version = m?.[1] ?? line; + } catch { + this.#version = undefined; + } + return this.#version; + } + + async listSessions(opts?: ListSessionsOptions): Promise { + let refs = await discoverSessions(); + if (opts?.projectDir) { + refs = refs.filter((r) => sameProjectDir(r.projectDir, opts.projectDir)); + } + const live = await liveStatuses(); + return refs.map((r) => { + r.status = live.get(r.id) ?? "idle"; + return r; + }); + } + + async listChildren(sessionId: string): Promise { + const kids = await childrenOf(sessionId); + const live = await liveStatuses(); + return kids.map((r) => { + r.status = live.get(r.id) ?? "idle"; + return r; + }); + } + + async getSession(sessionId: string): Promise { + const ref = await getSessionRow(sessionId); + if (!ref) return undefined; + const live = await liveStatuses(); + ref.status = live.get(sessionId) ?? "idle"; + return ref; + } + + async getTranscript(sessionId: string): Promise { + return readMuseTranscript(sessionId); + } + + async getTouchedFiles(sessionId: string): Promise { + return listSessionFiles(sessionId); + } + + async listAgents(_opts?: { projectDir?: string }): Promise { + return listMuseAgents(); + } + + async inject( + target: InjectTarget, + payload: ContextPayload, + opts: InjectOptions, + ): Promise { + return injectMuse(target, payload, opts.kickoffPrompt); + } + + async liveStatuses(): Promise> { + return liveStatuses(); + } +} diff --git a/packages/server/src/providers/muse/inject.ts b/packages/server/src/providers/muse/inject.ts new file mode 100644 index 0000000..d338517 --- /dev/null +++ b/packages/server/src/providers/muse/inject.ts @@ -0,0 +1,528 @@ +import { randomUUID } from "node:crypto"; +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import { execa } from "execa"; +import type { ContextPayload, InjectResult, InjectTarget } from "@threadle/shared"; +import { formatDuration, shortId } from "../run-metrics.js"; +import { resolveRunHarnessExtras, logHarnessExtrasLanes } from "../harness-extras.js"; +import { forEachLine, type LogLane, type LogSink } from "../stream.js"; +import { museBin } from "./paths.js"; + +const INJECT_TIMEOUT_MS = 10 * 60_000; + +/** + * Muse experimental "lane reminder gates". Setting each to `0`/`false` turns + * that reminder family off for the process (skill / goal / verify / …). + * When unset, Muse runs them and `muse exec` waits on the child sessions. + */ +export const MUSE_REMINDER_GATE_ENV = [ + "MUSE_EXPERIMENTAL_SKILL_REMINDER", + "MUSE_EXPERIMENTAL_GOAL_REMINDER", + "MUSE_EXPERIMENTAL_VERIFY_REMINDER", + "MUSE_EXPERIMENTAL_MEMORY_REMINDER", + "MUSE_EXPERIMENTAL_TODO_REMINDER", + "MUSE_EXPERIMENTAL_SCOPE_REMINDER", +] as const; + +/** Env for `muse exec`: when reminders are off, pin the gate vars to `0`. */ +export function museExecEnv(reminders: boolean): NodeJS.ProcessEnv { + const env = { ...process.env }; + if (!reminders) { + for (const key of MUSE_REMINDER_GATE_ENV) env[key] = "0"; + } + return env; +} + +const FALLBACK_MODELS = [ + "muse-spark-1.3", + "muse-spark-1.3-contributor", + "muse-spark-1.2", +]; + +/** Pull Spark model ids from muse help text (soft catalog until upstream ships `muse models`). */ +export function parseMuseSparkIds(helpText: string): string[] { + const ids = new Set(); + for (const m of helpText.matchAll(/muse-spark-[\w.-]+/g)) { + ids.add(m[0]!); + } + return [...ids]; +} + +async function helpText(args: string[]): Promise { + const { stdout, stderr } = await execa(museBin(), args, { + timeout: 8_000, + reject: false, + }); + return `${stdout}\n${stderr}`; +} + +export async function listMuseModels(): Promise { + // Prefer env override for labs. Upstream has no stable `muse models` yet. + const env = process.env.MUSE_MODELS?.trim(); + if (env) { + return env + .split(/[,:\s]+/) + .map((s) => s.trim()) + .filter(Boolean); + } + const ids = new Set(FALLBACK_MODELS); + try { + // Prefer exec help (inject surface); fall back to top-level help. + for (const args of [["exec", "--help"], ["--help"]] as string[][]) { + const text = await helpText(args); + for (const id of parseMuseSparkIds(text)) ids.add(id); + if (ids.size > FALLBACK_MODELS.length) break; + } + } catch { + /* keep fallbacks */ + } + return [...ids]; +} + +function contextText(payload: ContextPayload, kickoff?: string): string { + const parts = [ + "# Context handed off from another agent session", + "", + `(source: ${payload.source.provider} session ${payload.source.sessionId}${payload.meta.sourceTitle ? ` — "${payload.meta.sourceTitle}"` : ""}, payload kind: ${payload.kind})`, + "", + payload.content, + ]; + if (kickoff?.trim()) parts.push("", "---", "", kickoff.trim()); + return parts.join("\n"); +} + +function execArgs(opts: { + sessionId?: string; + model?: string; + promptFile: string; + /** Exclude foreign personal rules/skills (workspace still empty when bare). */ + ignoreLocalMarkdown?: boolean; +}): string[] { + const args = [ + "exec", + "--json", + "--prompt-file", + opts.promptFile, + "--approval-mode", + "never", + ]; + if (opts.ignoreLocalMarkdown) args.push("--no-foreign-personal-context"); + if (opts.model) args.push("--model", opts.model); + if (opts.sessionId) args.push("--session-id", opts.sessionId); + return args; +} + +/** Muse `recorded_at` is microseconds since epoch; older fixtures may use ms. */ +export function museRecordedMs(recordedAt: unknown): number | undefined { + if (typeof recordedAt !== "number" || !Number.isFinite(recordedAt)) return undefined; + return recordedAt > 1e14 ? recordedAt / 1000 : recordedAt; +} + +function prettyTaskKind(taskKind: string): string { + // reminder.agent.skill-reminder → skill-reminder + const m = taskKind.match(/reminder\.agent\.(.+)$/); + if (m?.[1]) return m[1]; + if (taskKind === "model.meta.response") return "model"; + return taskKind; +} + +export type MuseIngestState = { + sessionId?: string; + resultText?: string; + deltas: string[]; + /** Wall-clock start (Date.now) — Muse recorded_at is often clustered / unusable for deltas. */ + wall0: number; + firstAnswerAt?: number; + modelId?: string; + tokensIn: number; + tokensOut: number; + /** task_id → display name + start time (wall) */ + tasks: Map; + spawnNames: string[]; +}; + +export function createMuseIngestState(now = Date.now()): MuseIngestState { + return { + deltas: [], + wall0: now, + tokensIn: 0, + tokensOut: 0, + tasks: new Map(), + spawnNames: [], + }; +} + +/** Accumulate input/output tokens from a Muse usage object (any nesting). */ +function harvestMuseUsage(state: MuseIngestState, usage: unknown): void { + if (!usage || typeof usage !== "object" || Array.isArray(usage)) return; + const u = usage as Record; + if (typeof u.input_tokens === "number") state.tokensIn += u.input_tokens; + if (typeof u.output_tokens === "number") state.tokensOut += u.output_tokens; +} + +function relativeLabel(state: MuseIngestState, at: number): string { + return `@${formatDuration(at - state.wall0)}`; +} + +/** + * Pull session id + assistant text from one muse `--json` MSP line, and emit + * run-log lanes (text / tool / meta). Unknown payload types are skipped. + */ +export function ingestMuseJsonLine( + line: string, + state: MuseIngestState, +): Array<[LogLane, string]> { + let row: Record; + try { + row = JSON.parse(line) as Record; + } catch { + return []; + } + + const at = Date.now(); + + if (typeof row.session_id === "string" && row.session_id) { + state.sessionId = row.session_id; + } else if (typeof row.sessionId === "string" && row.sessionId) { + state.sessionId = row.sessionId; + } + const stream = row.stream; + if (stream && typeof stream === "object" && !Array.isArray(stream)) { + const s = stream as Record; + if (s.kind === "session" && typeof s.id === "string" && s.id) { + state.sessionId = s.id; + } + } + + const pt = typeof row.payload_type === "string" ? row.payload_type : ""; + const payload = + row.payload && typeof row.payload === "object" && !Array.isArray(row.payload) + ? (row.payload as Record) + : undefined; + if (!payload) return []; + + const out: Array<[LogLane, string]> = []; + + if (pt === "run.model.configured") { + const model = + (typeof payload.model_id === "string" && payload.model_id) || + (typeof payload.display_label === "string" && payload.display_label) || + undefined; + if (model) { + state.modelId = model; + out.push(["meta", `model ${model}`]); + } + return out; + } + + if (pt === "run.lifecycle.started") { + out.push(["meta", `run started${relativeLabel(state, at)}`]); + return out; + } + + // Session/runtime events may carry model_completed.usage (tokens). + if (pt === "runtime.session" || pt.startsWith("runtime.")) { + const event = + payload.event && typeof payload.event === "object" && !Array.isArray(payload.event) + ? (payload.event as Record) + : undefined; + if (event?.kind === "model_completed") { + harvestMuseUsage(state, event.usage); + if (typeof event.model === "string" && event.model) state.modelId = event.model; + } + harvestMuseUsage(state, payload.usage); + return out; + } + + if (pt === "run.output.delta" && typeof payload.text === "string" && payload.text) { + state.deltas.push(payload.text); + if (state.firstAnswerAt === undefined) { + state.firstAnswerAt = at; + out.push(["meta", `◷ first answer ${relativeLabel(state, at)}`]); + } + out.push(["text", payload.text]); + return out; + } + + if (pt === "run.terminal.completed") { + if (typeof payload.text === "string" && payload.text.trim()) { + state.resultText = payload.text.trim(); + } + out.push(["meta", museRunSummary(state, at)]); + return out; + } + + if (pt.startsWith("task.lifecycle.")) { + const taskId = typeof payload.task_id === "string" ? payload.task_id : undefined; + const event = + payload.event && typeof payload.event === "object" && !Array.isArray(payload.event) + ? (payload.event as Record) + : undefined; + if (!taskId || !event) return out; + const ek = typeof event.kind === "string" ? event.kind : ""; + + if (ek === "proposed" && typeof event.task_kind === "string") { + const name = prettyTaskKind(event.task_kind); + state.tasks.set(taskId, { name }); + return out; + } + + if (ek === "started") { + const task = state.tasks.get(taskId) ?? { name: shortId(taskId, 8) }; + task.startedAt = at; + state.tasks.set(taskId, task); + // Skip noisy model.meta.response start — first-answer meta covers it. + if (task.name === "model") return out; + if (!task.spawned) { + task.spawned = true; + if (!state.spawnNames.includes(task.name)) state.spawnNames.push(task.name); + out.push(["meta", `⎇ spawn ${task.name}${relativeLabel(state, at)}`]); + // Non-reminder child tasks surface as a tool lane for live phase HUD. + if (!/reminder/i.test(task.name)) { + out.push(["tool", task.name]); + } + } + return out; + } + + if (ek === "completed") { + const task = state.tasks.get(taskId); + if (!task || task.name === "model") return out; + const dur = + task.startedAt !== undefined + ? formatDuration(at - task.startedAt) + : undefined; + out.push(["meta", `✓ ${task.name}${dur ? ` ${dur}` : ""}`.trim()]); + return out; + } + + if (ek === "status" && typeof event.message === "string" && event.message) { + const msg = event.message; + // Keep only phase transitions that explain wall time (retries / stream). + if (/attempt|retry|error|fail|succeed|completed|opening/i.test(msg)) { + const task = state.tasks.get(taskId); + if (task?.name === "model") { + out.push(["meta", msg]); + } + } + return out; + } + + if (ek === "side_effect_intent") { + const op = typeof event.operation === "string" ? event.operation : ""; + if (op === "reminder.child_run") { + const task = state.tasks.get(taskId); + const key = + typeof event.idempotency_key === "string" ? event.idempotency_key : undefined; + // Prefer name from proposed; else scrape reminder_child:NAME:n + if (task && !task.spawned) { + const fromKey = key?.match(/reminder_child:([^:]+)/)?.[1]; + if (fromKey && task.name.startsWith("01")) task.name = fromKey; + } + } + return out; + } + } + + return out; +} + +/** End-of-run one-liner from accumulated muse stream state. */ +export function museRunSummary(state: MuseIngestState, endAt = Date.now()): string { + const parts: string[] = [formatDuration(endAt - state.wall0)]; + if (state.firstAnswerAt !== undefined) { + parts.push(`answer@${formatDuration(state.firstAnswerAt - state.wall0)}`); + if (endAt > state.firstAnswerAt + 250) { + parts.push(`post-answer ${formatDuration(endAt - state.firstAnswerAt)}`); + } + } + if (state.tokensIn || state.tokensOut) { + parts.push(`${state.tokensIn}→${state.tokensOut} tok`); + } + if (state.spawnNames.length) { + parts.push(`spawns ${state.spawnNames.length} (${state.spawnNames.join(", ")})`); + } + if (state.modelId) parts.push(state.modelId); + const sid = shortId(state.sessionId); + if (sid) parts.push(`session ${sid}`); + return `■ ${parts.join(" · ")}`; +} + +async function withPromptFile( + prompt: string, + fn: (file: string) => Promise, +): Promise { + const file = path.join(os.tmpdir(), `threadle-muse-${randomUUID()}.txt`); + await fs.promises.writeFile(file, prompt, "utf8"); + try { + return await fn(file); + } finally { + await fs.promises.unlink(file).catch(() => undefined); + } +} + +type MuseStreamResult = { + sessionId?: string; + resultText?: string; + isError: boolean; + usage?: { inputTokens: number; outputTokens: number }; +}; + +function museUsageFromState( + state: MuseIngestState, +): { inputTokens: number; outputTokens: number } | undefined { + if (!state.tokensIn && !state.tokensOut) return undefined; + return { inputTokens: state.tokensIn, outputTokens: state.tokensOut }; +} + +async function runMuseStreaming( + args: string[], + cwd: string, + onLog?: LogSink, + signal?: AbortSignal, + reminders = false, +): Promise { + const wall0 = Date.now(); + const child = execa(museBin(), args, { + cwd, + timeout: INJECT_TIMEOUT_MS, + stdin: "ignore", + cancelSignal: signal, + reject: false, + env: museExecEnv(reminders), + }); + const state = createMuseIngestState(wall0); + if (onLog) { + logHarnessExtrasLanes(onLog, "muse", reminders); + } + let sawTerminal = false; + forEachLine(child.stdout, (line) => { + const logs = ingestMuseJsonLine(line, state); + if (logs.some(([lane, text]) => lane === "meta" && text.startsWith("■"))) { + sawTerminal = true; + } + if (onLog) for (const [lane, text] of logs) onLog(lane, text); + }); + forEachLine(child.stderr, (line) => { + if (onLog) onLog("raw" as LogLane, line); + }); + const result = await child; + const sidIdx = args.indexOf("--session-id"); + if (sidIdx >= 0 && args[sidIdx + 1]) state.sessionId = state.sessionId ?? args[sidIdx + 1]; + const fromDeltas = state.deltas.join("").trim(); + if (onLog && !sawTerminal) { + onLog("meta", museRunSummary(state)); + } + return { + sessionId: state.sessionId, + resultText: state.resultText || fromDeltas || undefined, + isError: result.exitCode !== 0 && result.exitCode !== undefined, + usage: museUsageFromState(state), + }; +} + +async function harnessExtrasEnabled(): Promise { + return resolveRunHarnessExtras(); +} + +export async function injectMuse( + target: InjectTarget, + payload: ContextPayload, + kickoff?: string, +): Promise { + const prompt = contextText(payload, kickoff); + const cwd = target.projectDir || process.cwd(); + + if (target.mode === "continue") { + if (!target.sessionId) throw new Error("continue inject requires a sessionId"); + const reminders = await harnessExtrasEnabled(); + const r = await withPromptFile(prompt, (file) => + runMuseStreaming( + execArgs({ promptFile: file, model: target.model, sessionId: target.sessionId }), + cwd, + undefined, + undefined, + reminders, + ), + ); + if (r.isError) throw new Error("muse continue inject failed"); + return { + newSessionId: r.sessionId ?? target.sessionId, + provider: "muse", + resultText: r.resultText, + usage: r.usage, + }; + } + + if (target.mode === "new-session") { + const pinned = randomUUID(); + const reminders = await harnessExtrasEnabled(); + const r = await withPromptFile(prompt, (file) => + runMuseStreaming( + execArgs({ promptFile: file, model: target.model, sessionId: pinned }), + cwd, + undefined, + undefined, + reminders, + ), + ); + if (r.isError) throw new Error("muse new-session inject failed"); + return { + newSessionId: r.sessionId ?? pinned, + provider: "muse", + resultText: r.resultText, + usage: r.usage, + }; + } + + throw new Error(`unsupported inject mode for muse: ${target.mode}`); +} + +export async function runMuseAgent(opts: { + agent?: string; + model?: string; + prompt: string; + projectDir: string; + sessionId?: string; + onLog?: LogSink; + signal?: AbortSignal; + /** Override Settings `harnessExtras` for this spawn. */ + harnessExtras?: boolean; + /** @deprecated */ + reminders?: boolean; + ignoreLocalMarkdown?: boolean; +}): Promise { + const pinned = opts.sessionId ?? randomUUID(); + const reminders = + opts.harnessExtras !== undefined + ? opts.harnessExtras + : opts.reminders !== undefined + ? opts.reminders + : await harnessExtrasEnabled(); + if (opts.onLog && opts.ignoreLocalMarkdown) { + opts.onLog("meta", "ignore local md · bare cwd + --no-foreign-personal-context"); + } + const r = await withPromptFile(opts.prompt, (file) => + runMuseStreaming( + execArgs({ + promptFile: file, + model: opts.model, + sessionId: pinned, + ignoreLocalMarkdown: opts.ignoreLocalMarkdown, + }), + opts.projectDir, + opts.onLog, + opts.signal, + reminders, + ), + ); + if (r.isError) throw new Error("muse agent run failed"); + const id = r.sessionId ?? pinned; + return { + newSessionId: id, + provider: "muse", + resultText: r.resultText, + usage: r.usage, + }; +} diff --git a/packages/server/src/providers/muse/paths.ts b/packages/server/src/providers/muse/paths.ts new file mode 100644 index 0000000..67c6449 --- /dev/null +++ b/packages/server/src/providers/muse/paths.ts @@ -0,0 +1,72 @@ +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import { resolveCliBin } from "../../platform-bin.js"; + +/** Muse Code config/auth home: `$MUSE_HOME` or `~/.muse`. Never write here. */ +export function museConfigHome(): string { + const env = process.env.MUSE_HOME?.trim(); + if (env) return path.resolve(env); + return path.join(os.homedir(), ".muse"); +} + +/** + * Muse data share (sessions, skills, plugins, runtime). + * `$MUSE_DATA_DIR` or `$XDG_DATA_HOME/muse` or `~/.local/share/muse`. + */ +export function museShare(): string { + const env = process.env.MUSE_DATA_DIR?.trim(); + if (env) return path.resolve(env); + const xdg = process.env.XDG_DATA_HOME?.trim(); + if (xdg) return path.join(path.resolve(xdg), "muse"); + return path.join(os.homedir(), ".local", "share", "muse"); +} + +export function sessionsRoot(): string { + return path.join(museShare(), "sessions"); +} + +export function sessionIndexDb(): string { + return path.join(museShare(), "session-index.db"); +} + +export function runtimeSessionsDir(): string { + return path.join(museShare(), "runtime", "muse", "sessions"); +} + +export function skillsRoot(): string { + return path.join(museShare(), "skills"); +} + +export function pluginsRoot(): string { + return path.join(museShare(), "plugins"); +} + +let cachedBin: string | undefined; + +/** Resolve the `muse` CLI binary. */ +export function museBin(): string { + if (cachedBin) return cachedBin; + const fromEnv = process.env.MUSE_PATH?.trim(); + if (fromEnv) { + cachedBin = fromEnv; + return cachedBin; + } + cachedBin = resolveCliBin("muse", { + extraUnix: [path.join(os.homedir(), ".local", "bin", "muse")], + }); + return cachedBin; +} + +export function resetMuseBinCache(): void { + cachedBin = undefined; +} + +export function sessionsRootAvailable(): boolean { + try { + fs.accessSync(sessionsRoot(), fs.constants.R_OK); + return true; + } catch { + return false; + } +} diff --git a/packages/server/src/providers/muse/transcript.ts b/packages/server/src/providers/muse/transcript.ts new file mode 100644 index 0000000..982c7c6 --- /dev/null +++ b/packages/server/src/providers/muse/transcript.ts @@ -0,0 +1,153 @@ +import fs from "node:fs"; +import type { ContentPart, NormalizedMessage } from "@threadle/shared"; +import { asObj, recordsFromLine, resolveLogPath, microsToMs } from "./discover.js"; + +/** + * Parse Muse MSP session.jsonl into NormalizedMessage. + * Known: user intents, assistant_message_committed, reasoning_committed, tool events. + * Unknown payload/event kinds are skipped. + */ +export async function readMuseTranscript(sessionId: string): Promise { + const logPath = await resolveLogPath(sessionId); + if (!logPath) return []; + let raw: string; + try { + raw = await fs.promises.readFile(logPath, "utf8"); + } catch { + return []; + } + + const out: NormalizedMessage[] = []; + let idx = 0; + const seenAssistant = new Set(); + + for (const line of raw.split("\n")) { + if (!line.trim()) continue; + for (const rj of recordsFromLine(line)) { + const pt = typeof rj.payload_type === "string" ? rj.payload_type : ""; + const ts = microsToMs(rj.recorded_at) || undefined; + const payload = asObj(rj.payload) ?? {}; + + if (pt === "runtime.user_intent.accepted") { + const texts: string[] = []; + const blocks = payload.refill_blocks; + if (Array.isArray(blocks)) { + for (const b of blocks) { + const t = asObj(b)?.text; + if (typeof t === "string" && t.trim()) texts.push(t.trim()); + } + } + if (!texts.length) { + const msgs = payload.model_messages; + if (Array.isArray(msgs)) { + for (const m of msgs) { + const content = asObj(m)?.content; + if (!Array.isArray(content)) continue; + for (const c of content) { + const t = asObj(c)?.text; + if (typeof t === "string" && t.trim()) texts.push(t.trim()); + } + } + } + } + const text = texts.join("\n").trim(); + if (!text) continue; + out.push({ + id: typeof payload.intent_id === "string" ? payload.intent_id : `${sessionId}:u:${idx++}`, + role: "user", + parts: [{ type: "text", text }], + timestamp: ts, + }); + continue; + } + + if (pt === "runtime.session" && payload.kind === "run") { + const event = asObj(payload.event); + if (!event) continue; + const kind = typeof event.kind === "string" ? event.kind : ""; + + if (kind === "assistant_message_committed") { + const text = typeof event.text === "string" ? event.text : ""; + const mid = + typeof event.message_id === "string" ? event.message_id : `${sessionId}:a:${idx++}`; + if (seenAssistant.has(mid)) continue; + seenAssistant.add(mid); + if (!text.trim()) continue; + out.push({ + id: mid, + role: "assistant", + parts: [{ type: "text", text }], + timestamp: ts, + }); + continue; + } + + if (kind === "reasoning_committed") { + const text = typeof event.text === "string" ? event.text : ""; + if (!text.trim()) continue; + const mid = + typeof event.message_id === "string" + ? `${event.message_id}:think` + : `${sessionId}:t:${idx++}`; + out.push({ + id: mid, + role: "assistant", + parts: [{ type: "thinking", text }], + timestamp: ts, + }); + continue; + } + + if (kind === "model_completed") { + const usage = asObj(event.usage); + if (!usage || out.length === 0) continue; + const last = out[out.length - 1]; + if (last?.role === "assistant") { + last.tokens = { + input: typeof usage.input_tokens === "number" ? usage.input_tokens : undefined, + output: typeof usage.output_tokens === "number" ? usage.output_tokens : undefined, + cacheRead: + typeof usage.cache_read_tokens === "number" + ? usage.cache_read_tokens + : typeof usage.cached_tokens === "number" + ? usage.cached_tokens + : undefined, + cacheWrite: + typeof usage.cache_write_tokens === "number" + ? usage.cache_write_tokens + : undefined, + }; + } + continue; + } + + // Tool calls (best-effort): tool_* event kinds with name + input + if (kind.startsWith("tool_") || kind.includes("tool_call")) { + const toolName = + (typeof event.tool_name === "string" && event.tool_name) || + (typeof event.name === "string" && event.name) || + undefined; + if (!toolName) continue; + const parts: ContentPart[] = [ + { + type: kind.includes("result") ? "tool_result" : "tool_use", + toolName, + toolInput: event.input ?? event.args ?? event.arguments, + toolUseId: typeof event.tool_use_id === "string" ? event.tool_use_id : undefined, + text: typeof event.text === "string" ? event.text : undefined, + isError: event.is_error === true, + }, + ]; + out.push({ + id: `${sessionId}:tool:${idx++}`, + role: "assistant", + parts, + timestamp: ts, + }); + } + } + } + } + + return out; +} diff --git a/packages/server/src/providers/opencode/inject.ts b/packages/server/src/providers/opencode/inject.ts index 8389578..1a89da5 100644 --- a/packages/server/src/providers/opencode/inject.ts +++ b/packages/server/src/providers/opencode/inject.ts @@ -1,8 +1,10 @@ import crypto from "node:crypto"; import { execa, type ResultPromise } from "execa"; +import { estimateTokenBaseline, formatBaselineSuffix } from "@threadle/shared"; import type { ContextPayload, InjectResult, InjectTarget } from "@threadle/shared"; import { forEachLine, toolSummary, type LogLane, type LogSink } from "../stream.js"; import { positionalSafe } from "../argv-safe.js"; +import { query, type SessionRow } from "./db.js"; const INJECT_TIMEOUT_MS = 10 * 60_000; @@ -58,6 +60,58 @@ function findResultText(stdout: string): string | undefined { return text.trim() ? text : undefined; } +/** Accumulate step-finish tokens from an opencode JSON event stream. */ +export function findOpencodeUsageInJsonl(stdout: string): { + inputTokens: number; + outputTokens: number; +} | undefined { + let inputTokens = 0; + let outputTokens = 0; + let saw = false; + for (const line of stdout.split("\n")) { + if (!line.trim()) continue; + try { + const evt = JSON.parse(line) as Record; + let bag: unknown; + if (evt.type === "step-finish" && evt.tokens) { + bag = evt.tokens; + } else { + const part = + evt.part && typeof evt.part === "object" && !Array.isArray(evt.part) + ? (evt.part as Record) + : undefined; + if (part?.type === "step-finish" && part.tokens) bag = part.tokens; + } + if (!bag || typeof bag !== "object" || Array.isArray(bag)) continue; + const t = bag as Record; + const inn = + typeof t.input === "number" + ? t.input + : typeof t.input_tokens === "number" + ? t.input_tokens + : undefined; + const out = + typeof t.output === "number" + ? t.output + : typeof t.output_tokens === "number" + ? t.output_tokens + : undefined; + if (inn != null) { + inputTokens += inn; + saw = true; + } + if (out != null) { + outputTokens += out; + saw = true; + } + } catch { + // skip + } + } + if (!saw || (!inputTokens && !outputTokens)) return undefined; + return { inputTokens, outputTokens }; +} + function scanForSessionId(value: unknown, depth: number): string | undefined { if (depth > 4 || !value || typeof value !== "object") return undefined; const obj = value as Record; @@ -286,6 +340,59 @@ export interface RunAgentOptions { /** live stream of the run's text/tool events */ onLog?: LogSink; signal?: AbortSignal; + ignoreLocalMarkdown?: boolean; +} + +async function sessionTokensFromDb( + sessionId: string, +): Promise<{ inputTokens: number; outputTokens: number } | undefined> { + try { + const rows = await query( + "SELECT tokens_input, tokens_output FROM session WHERE id = ?", + [sessionId], + ); + const row = rows[0]; + if (!row) return undefined; + const inputTokens = Number(row.tokens_input) || 0; + const outputTokens = Number(row.tokens_output) || 0; + if (!inputTokens && !outputTokens) return undefined; + return { inputTokens, outputTokens }; + } catch { + return undefined; + } +} + +async function waitSessionTokensFromDb( + sessionId: string, + attempts = 4, + delayMs = 150, +): Promise<{ inputTokens: number; outputTokens: number } | undefined> { + for (let i = 0; i < attempts; i++) { + const u = await sessionTokensFromDb(sessionId); + if (u) return u; + if (i + 1 < attempts) { + await new Promise((r) => setTimeout(r, delayMs * (i + 1))); + } + } + return undefined; +} + +function logTokMeta( + onLog: LogSink | undefined, + usage: { inputTokens: number; outputTokens: number } | undefined, + prompt: string | undefined, + ignoreLocalMarkdown: boolean | undefined, +): void { + if (!onLog || !usage) return; + const parts = [`${usage.inputTokens}→${usage.outputTokens} tok`]; + if (prompt) { + const b = estimateTokenBaseline(usage.inputTokens, prompt, { + ignoreLocalMarkdown, + }); + const suf = b ? formatBaselineSuffix(b) : ""; + if (suf) parts.push(suf); + } + onLog("meta", `■ ${parts.join(" · ")}`); } /** Map one `opencode run --format json` event line onto a log lane. */ @@ -323,28 +430,37 @@ export async function runOpencodeAgent(opts: RunAgentOptions): Promise { + const lines: string[] = []; + forEachLine(child.stdout, (line) => { + lines.push(line); + if (opts.onLog) { const log = opencodeEventToLog(line); - if (log) onLog(log[0], log[1]); - }); - } - const { stdout } = await child; - const newSessionId = findSessionId(stdout) ?? opts.sessionId; + if (log) opts.onLog(log[0], log[1]); + } + }); + await child; + const streamText = lines.join("\n"); + const newSessionId = findSessionId(streamText) ?? opts.sessionId; if (!newSessionId) { throw new Error("could not determine session id from opencode run output"); } + let usage = findOpencodeUsageInJsonl(streamText); + if (!usage) usage = await waitSessionTokensFromDb(newSessionId); + logTokMeta(opts.onLog, usage, opts.prompt, opts.ignoreLocalMarkdown); return { newSessionId, provider: "opencode", - resultText: findResultText(stdout), + resultText: findResultText(streamText), + usage, }; } diff --git a/packages/server/src/providers/pilot-defaults.ts b/packages/server/src/providers/pilot-defaults.ts new file mode 100644 index 0000000..9a7079d --- /dev/null +++ b/packages/server/src/providers/pilot-defaults.ts @@ -0,0 +1,187 @@ +import type { ProviderId } from "@threadle/shared"; +import { PROVIDER_HARNESS_EXTRAS } from "@threadle/shared"; + +export const PILOT_PROMPT = "Reply with exactly the single word: ok"; + +/** One smoke spawn in the Settings test-pilot matrix */ +export interface PilotCaseDef { + id: string; + provider: ProviderId; + /** Short case label (table “case” column) */ + variant: string; + agent: string; + model?: string; + permissionMode?: string; + sandbox?: string; + askForApproval?: string; + /** Override Settings harnessExtras for this spawn */ + harnessExtras?: boolean; + /** Only run when Settings pilot “extra tests” is on */ + extra?: boolean; + note?: string; +} + +const BASE: Record< + ProviderId, + Omit +> = { + "claude-code": { agent: "general-purpose", model: "haiku", harnessExtras: false }, + cursor: { + agent: "ask", + model: "auto", + harnessExtras: false, + note: "ask mode (read-only)", + }, + antigravity: { agent: "agent", harnessExtras: false }, + codex: { agent: "codex", harnessExtras: false }, + copilot: { agent: "copilot", harnessExtras: false }, + grok: { agent: "grok", model: "grok-4.5", harnessExtras: false }, + muse: { agent: "muse", harnessExtras: false }, + opencode: { agent: "build", harnessExtras: false }, +}; + +const HARNESS_PROVIDERS = Object.keys(PROVIDER_HARNESS_EXTRAS) as ProviderId[]; + +/** Pair of extras:off / extras:on for every provider with a skip switch */ +function harnessExtraCases(): PilotCaseDef[] { + const out: PilotCaseDef[] = []; + for (const provider of HARNESS_PROVIDERS) { + const base = BASE[provider]; + const lanes = PROVIDER_HARNESS_EXTRAS[provider]?.lanes ?? []; + const laneNote = lanes.join(", "); + out.push({ + id: `${provider}:extras-off`, + provider, + variant: "extras:off", + agent: base.agent, + model: base.model, + harnessExtras: false, + extra: true, + note: laneNote ? `skip: ${laneNote}` : "harness extras off", + }); + out.push({ + id: `${provider}:extras-on`, + provider, + variant: "extras:on", + agent: base.agent, + model: base.model, + harnessExtras: true, + extra: true, + note: laneNote ? `run: ${laneNote}` : "harness extras on", + }); + } + return out; +} + +/** Optional flag / mode matrix — toggled on in the pilot UI */ +const EXTRA: PilotCaseDef[] = [ + { + id: "claude-code:permission-plan", + provider: "claude-code", + variant: "permission:plan", + agent: "general-purpose", + model: "haiku", + permissionMode: "plan", + harnessExtras: false, + extra: true, + note: "--permission-mode plan", + }, + { + id: "claude-code:permission-accept", + provider: "claude-code", + variant: "permission:acceptEdits", + agent: "general-purpose", + model: "haiku", + permissionMode: "acceptEdits", + harnessExtras: false, + extra: true, + note: "--permission-mode acceptEdits", + }, + { + id: "cursor:agent", + provider: "cursor", + variant: "mode:agent", + agent: "agent", + model: "auto", + harnessExtras: false, + extra: true, + note: "full agent mode", + }, + { + id: "cursor:plan", + provider: "cursor", + variant: "mode:plan", + agent: "plan", + model: "auto", + harnessExtras: false, + extra: true, + note: "--mode plan", + }, + { + id: "antigravity:mode-plan", + provider: "antigravity", + variant: "mode:plan", + agent: "plan", + harnessExtras: false, + extra: true, + note: "--mode plan", + }, + { + id: "codex:sandbox-ro", + provider: "codex", + variant: "sandbox:read-only", + agent: "codex", + sandbox: "read-only", + askForApproval: "never", + harnessExtras: false, + extra: true, + note: "--sandbox read-only", + }, + { + id: "codex:sandbox-full", + provider: "codex", + variant: "sandbox:danger-full-access", + agent: "codex", + sandbox: "danger-full-access", + askForApproval: "never", + harnessExtras: false, + extra: true, + note: "--sandbox danger-full-access", + }, + { + id: "codex:ask-on-request", + provider: "codex", + variant: "ask:on-request", + agent: "codex", + sandbox: "workspace-write", + askForApproval: "on-request", + harnessExtras: false, + extra: true, + note: "sandbox without --approve-for-me", + }, + ...harnessExtraCases(), +]; + +function baseCase(provider: ProviderId): PilotCaseDef { + const b = BASE[provider]; + return { + id: provider, + provider, + variant: "default", + ...b, + extra: false, + }; +} + +export const PILOT_PROVIDER_IDS = Object.keys(BASE) as ProviderId[]; + +/** All pilot cases (base + optional extras) */ +export function allPilotCases(): PilotCaseDef[] { + return [...PILOT_PROVIDER_IDS.map(baseCase), ...EXTRA]; +} + +/** Cases visible for the current extra-tests toggle */ +export function pilotCasesFor(extraTests: boolean): PilotCaseDef[] { + const all = allPilotCases(); + return extraTests ? all : all.filter((c) => !c.extra); +} diff --git a/packages/server/src/providers/registry.ts b/packages/server/src/providers/registry.ts index 4485c74..525aa5e 100644 --- a/packages/server/src/providers/registry.ts +++ b/packages/server/src/providers/registry.ts @@ -6,6 +6,7 @@ import { CodexProvider } from "./codex/index.js"; import { CopilotProvider } from "./copilot/index.js"; import { CursorProvider } from "./cursor/index.js"; import { GrokProvider } from "./grok/index.js"; +import { MuseProvider } from "./muse/index.js"; import { OpencodeProvider } from "./opencode/index.js"; export class ProviderRegistry { @@ -20,6 +21,7 @@ export class ProviderRegistry { ["codex", new CodexProvider()], ["copilot", new CopilotProvider()], ["grok", new GrokProvider()], + ["muse", new MuseProvider()], ]); } diff --git a/packages/server/src/providers/run-metrics.ts b/packages/server/src/providers/run-metrics.ts new file mode 100644 index 0000000..9daf466 --- /dev/null +++ b/packages/server/src/providers/run-metrics.ts @@ -0,0 +1,103 @@ +import type { LogSink } from "./stream.js"; + +/** Human-readable duration for run-log meta lines */ +export function formatDuration(ms: number): string { + if (!Number.isFinite(ms) || ms < 0) return "?"; + if (ms < 1000) return `${Math.round(ms)}ms`; + const s = ms / 1000; + if (s < 60) return `${s < 10 ? s.toFixed(1) : s.toFixed(0)}s`; + const m = Math.floor(s / 60); + const rem = Math.round(s - m * 60); + return `${m}m${rem.toString().padStart(2, "0")}s`; +} + +/** Short session id fragment for meta lines */ +export function shortId(id: string | undefined, n = 8): string { + if (!id) return ""; + return id.length <= n ? id : id.slice(0, n); +} + +export type RunTimer = { + readonly startedAt: number; + noteFirstAnswer: () => void; + noteSpawn: (name: string) => void; + noteTool: (name: string) => void; + /** Elapsed ms since start */ + elapsed: () => number; + /** One-line end summary (no leading glyph) */ + summary: (extra?: string) => string; +}; + +/** Wall-clock timer for provider / node runs */ +export function createRunTimer(): RunTimer { + const startedAt = Date.now(); + let firstAnswerAt: number | undefined; + const spawns: string[] = []; + let tools = 0; + + return { + startedAt, + noteFirstAnswer() { + if (firstAnswerAt === undefined) firstAnswerAt = Date.now(); + }, + noteSpawn(name: string) { + const n = name.trim(); + if (n && !spawns.includes(n)) spawns.push(n); + }, + noteTool(_name: string) { + tools += 1; + }, + elapsed() { + return Date.now() - startedAt; + }, + summary(extra?: string) { + const parts: string[] = [formatDuration(Date.now() - startedAt)]; + if (firstAnswerAt !== undefined) { + const toAnswer = firstAnswerAt - startedAt; + const post = Date.now() - firstAnswerAt; + parts.push(`answer@${formatDuration(toAnswer)}`); + if (post > 250) parts.push(`post-answer ${formatDuration(post)}`); + } + if (spawns.length) parts.push(`spawns ${spawns.length} (${spawns.join(", ")})`); + if (tools) parts.push(`${tools} tool call${tools === 1 ? "" : "s"}`); + if (extra?.trim()) parts.push(extra.trim()); + return parts.join(" · "); + }, + }; +} + +/** + * Wrap a provider agent call with ▶ / ■ meta lines (elapsed, optional session). + * Mid-run spawn/tool notes still come from the provider's own onLog. + */ +export async function withProviderRunMetrics( + opts: { + label: string; + provider: string; + model?: string; + onLog?: LogSink; + }, + run: (timer: RunTimer) => Promise, +): Promise { + const timer = createRunTimer(); + const head = [opts.provider, opts.model].filter(Boolean).join("/"); + opts.onLog?.( + "meta", + `▶ ${opts.label}${head ? ` (${head})` : ""}`, + ); + try { + const result = await run(timer); + const sid = shortId(result.newSessionId); + opts.onLog?.( + "meta", + `■ ${opts.label} ${timer.summary(sid ? `session ${sid}` : undefined)}`, + ); + return result; + } catch (err) { + opts.onLog?.( + "meta", + `■ ${opts.label} failed after ${formatDuration(timer.elapsed())}`, + ); + throw err; + } +} diff --git a/packages/server/src/providers/stream.ts b/packages/server/src/providers/stream.ts index 296523e..6f15c7c 100644 --- a/packages/server/src/providers/stream.ts +++ b/packages/server/src/providers/stream.ts @@ -1,7 +1,7 @@ import readline from "node:readline"; import type { Readable } from "node:stream"; -export type LogLane = "text" | "thinking" | "tool" | "raw"; +export type LogLane = "text" | "thinking" | "tool" | "raw" | "meta"; export type LogSink = (lane: LogLane, line: string) => void; /** Attach a per-line parser to a child stdout stream; parse errors are ignored. */ diff --git a/packages/server/src/routes/inject.ts b/packages/server/src/routes/inject.ts index 12d8d7b..df6d37f 100644 --- a/packages/server/src/routes/inject.ts +++ b/packages/server/src/routes/inject.ts @@ -13,7 +13,11 @@ injectRoutes.post("/", async (c) => { const payload = await readPayload(req.payloadHash); if (!payload) return c.json({ error: "payload not found — materialize first" }, 404); - const { jobId } = jobs.create("inject", req.target.provider); + const { jobId } = jobs.create("inject", req.target.provider, undefined, { + sessionRef: req.target.sessionId + ? { provider: req.target.provider, sessionId: req.target.sessionId } + : undefined, + }); void (async () => { try { diff --git a/packages/server/src/routes/internals.ts b/packages/server/src/routes/internals.ts index 192c4ab..78a4601 100644 --- a/packages/server/src/routes/internals.ts +++ b/packages/server/src/routes/internals.ts @@ -53,6 +53,14 @@ const ITEMS: ItemSpec[] = [ description: "Jump-list bookmarks for workflows, sessions, library payloads, skills, rules, and files.", clearable: false, }, + { + id: "pilot-sessions", + label: "Test pilot sessions", + rel: "pilot-sessions.json", + kind: "file", + description: "Session ids spawned by Settings → test pilot (show/hide in Sessions).", + clearable: false, + }, { id: "payloads", label: "Context payloads", rel: "payloads", kind: "dir", description: "Extracted summaries, transcript excerpts and their tags (content-addressed).", diff --git a/packages/server/src/routes/plugins.ts b/packages/server/src/routes/plugins.ts index a1f32b1..b250c12 100644 --- a/packages/server/src/routes/plugins.ts +++ b/packages/server/src/routes/plugins.ts @@ -6,6 +6,7 @@ import { codexHome } from "../providers/codex/paths.js"; import { copilotHome } from "../providers/copilot/paths.js"; import { cursorHome } from "../providers/cursor/paths.js"; import { grokHome } from "../providers/grok/paths.js"; +import { pluginsRoot as musePluginsRoot } from "../providers/muse/paths.js"; import { pathContained } from "../path-safe.js"; export type PluginProvider = @@ -13,7 +14,8 @@ export type PluginProvider = | "codex" | "copilot" | "grok" - | "cursor"; + | "cursor" + | "muse"; export type PluginOriginKind = | "marketplace-catalog" @@ -518,6 +520,39 @@ export async function listCursorPlugins(): Promise { return out; } +/** Muse plugin cache under ~/.local/share/muse/plugins (read-only). */ +export async function listMusePlugins(): Promise { + const roots = [ + path.join(musePluginsRoot(), "cache", "builtin"), + path.join(musePluginsRoot(), "installed"), + ]; + const out: PluginEntry[] = []; + for (const root of roots) { + let entries: fs.Dirent[]; + try { + entries = await fs.promises.readdir(root, { withFileTypes: true }); + } catch { + continue; + } + for (const e of entries) { + if (!e.isDirectory() || e.name.startsWith(".")) continue; + const pluginRoot = path.join(root, e.name); + out.push({ + provider: "muse", + id: e.name, + name: e.name, + origin: { + kind: root.includes("cache") ? "cache" : "installed-tree", + path: pluginRoot, + }, + state: root.includes("cache") ? "cached" : "installed", + children: await listPluginChildren(pluginRoot), + }); + } + } + return out; +} + export async function listAllPlugins( providerFilter?: string, ): Promise { @@ -530,6 +565,7 @@ export async function listAllPlugins( if (!want || want === "copilot") tasks.push(listCopilotPlugins()); if (!want || want === "grok") tasks.push(listGrokPlugins()); if (!want || want === "cursor") tasks.push(listCursorPlugins()); + if (!want || want === "muse") tasks.push(listMusePlugins()); const chunks = await Promise.all(tasks); const out = chunks.flat(); diff --git a/packages/server/src/routes/run.ts b/packages/server/src/routes/run.ts index 85d7fbd..f1f0b9b 100644 --- a/packages/server/src/routes/run.ts +++ b/packages/server/src/routes/run.ts @@ -1,14 +1,22 @@ import { Hono } from "hono"; import matter from "gray-matter"; -import type { InjectResult, ModelInfo, RunAgentRequest, RunSessionRequest } from "@threadle/shared"; +import type { + InjectResult, + ModelInfo, + RunAgentRequest, + RunSessionRequest, +} from "@threadle/shared"; import { claudeExhaustedForModel, enrichUsageExhaustionError, + estimateTokenBaseline, formatUsageExhaustionMessage, isAbsolutePath, summarizeGraphExecution, } from "@threadle/shared"; import { isKnownProjectDir } from "../readable-paths.js"; +import { ensureBareWorkspace } from "../providers/bare-workspace.js"; +import { markPilotSession } from "../pilot-sessions/store.js"; import { bus } from "../events.js"; import { appendJobLog, appLog, jobs, mergeJobNodeOutput, readAppLogs, readJobLogs, readJobOutputs, findLatestGraphOutputs, type CachedNodeOutput } from "../jobs.js"; import { @@ -24,11 +32,17 @@ import { import { listCodexModels, runCodexAgent } from "../providers/codex/inject.js"; import { listCopilotModels, runCopilotAgent } from "../providers/copilot/inject.js"; import { listGrokModels, runGrokAgent } from "../providers/grok/inject.js"; +import { listMuseModels, runMuseAgent } from "../providers/muse/inject.js"; import { registry } from "../providers/registry.js"; import { executeWorkflow } from "../workflows/executor.js"; import { readGraph } from "../graphs/store.js"; import { readSubscription } from "./subscription.js"; import type { LogSink } from "../providers/stream.js"; +import { + allPilotCases, + PILOT_PROMPT, + pilotCasesFor, +} from "../providers/pilot-defaults.js"; const LOG_LINE_MAX = 2000; const KNOWN_PROVIDERS = new Set([ @@ -39,6 +53,7 @@ const KNOWN_PROVIDERS = new Set([ "codex", "copilot", "grok", + "muse", ]); function logSinkFor(jobId: string): LogSink { @@ -52,6 +67,30 @@ function logSinkFor(jobId: string): LogSink { export const runRoutes = new Hono(); export const modelRoutes = new Hono(); +/** Cheap one-shot smoke plan for Settings → test pilot. */ +runRoutes.get("/pilot", async (c) => { + const extraTests = c.req.query("extra") === "1" || c.req.query("extra") === "true"; + const info = await registry.info(); + const byId = new Map(info.map((p) => [p.id, p])); + const cases = pilotCasesFor(extraTests).map((def) => { + const available = byId.get(def.provider)?.available === true; + return { + ...def, + available, + skipReason: available ? undefined : "CLI not available", + }; + }); + const extras = allPilotCases().filter((c) => c.extra); + return c.json({ + prompt: PILOT_PROMPT, + extraTests, + harnessExtras: false as const, + cases, + /** Catalog of optional cases (for UI copy) even when extraTests is off. */ + extraCaseCount: extras.length, + }); +}); + // aliases the claude CLI accepts for --model, plus room for full ids via free text const CLAUDE_MODELS = ["sonnet", "opus", "haiku", "claude-fable-5", "claude-opus-5", "claude-sonnet-5"]; @@ -78,6 +117,9 @@ modelRoutes.get("/", async (c) => { for (const id of await listGrokModels()) { models.push({ provider: "grok", id }); } + for (const id of await listMuseModels()) { + models.push({ provider: "muse", id }); + } return c.json(models); }); @@ -89,12 +131,25 @@ runRoutes.post("/agent", async (c) => { if (!KNOWN_PROVIDERS.has(req.provider)) { return c.json({ error: `unknown provider: ${String(req.provider)}` }, 400); } - if (req.projectDir && !(await isKnownProjectDir(req.projectDir))) { + if ( + !req.ignoreLocalMarkdown && + req.projectDir && + !(await isKnownProjectDir(req.projectDir)) + ) { return c.json({ error: `projectDir is not a known project directory: ${req.projectDir}` }, 403); } - const projectDir = req.projectDir || process.cwd(); + const projectDir = req.ignoreLocalMarkdown + ? await ensureBareWorkspace() + : req.projectDir || process.cwd(); - const { jobId, signal } = jobs.create("run-agent", `${req.provider}:${req.agent}`, req.graphId); + const { jobId, signal } = jobs.create( + "run-agent", + `${req.provider}:${req.agent}`, + req.graphId, + req.sessionId + ? { sessionRef: { provider: req.provider, sessionId: req.sessionId } } + : undefined, + ); void (async () => { try { bus.publish({ @@ -104,6 +159,9 @@ runRoutes.post("/agent", async (c) => { }); const onLog = logSinkFor(jobId); + if (req.ignoreLocalMarkdown) { + onLog("meta", `bare workspace · ignore local md (${projectDir})`); + } if (req.provider === "claude-code") { try { const sub = await readSubscription(); @@ -135,6 +193,7 @@ runRoutes.post("/agent", async (c) => { sessionId: req.sessionId, onLog, signal, + ignoreLocalMarkdown: req.ignoreLocalMarkdown, }); } else if (req.provider === "cursor") { result = await runCursorAgent({ @@ -145,6 +204,7 @@ runRoutes.post("/agent", async (c) => { sessionId: req.sessionId, onLog, signal, + ignoreLocalMarkdown: req.ignoreLocalMarkdown, }); } else if (req.provider === "antigravity") { result = await runAntigravityAgent({ @@ -155,6 +215,8 @@ runRoutes.post("/agent", async (c) => { sessionId: req.sessionId, onLog, signal, + harnessExtras: req.harnessExtras ?? req.museReminders, + ignoreLocalMarkdown: req.ignoreLocalMarkdown, }); } else if (req.provider === "codex") { result = await runCodexAgent({ @@ -167,6 +229,7 @@ runRoutes.post("/agent", async (c) => { askForApproval: req.askForApproval, onLog, signal, + ignoreLocalMarkdown: req.ignoreLocalMarkdown, }); } else if (req.provider === "copilot") { result = await runCopilotAgent({ @@ -177,6 +240,7 @@ runRoutes.post("/agent", async (c) => { sessionId: req.sessionId, onLog, signal, + ignoreLocalMarkdown: req.ignoreLocalMarkdown, }); } else if (req.provider === "grok") { result = await runGrokAgent({ @@ -187,6 +251,20 @@ runRoutes.post("/agent", async (c) => { sessionId: req.sessionId, onLog, signal, + harnessExtras: req.harnessExtras ?? req.museReminders, + ignoreLocalMarkdown: req.ignoreLocalMarkdown, + }); + } else if (req.provider === "muse") { + result = await runMuseAgent({ + agent: req.agent, + model: req.model, + prompt: req.prompt, + projectDir, + sessionId: req.sessionId, + onLog, + signal, + harnessExtras: req.harnessExtras ?? req.museReminders, + ignoreLocalMarkdown: req.ignoreLocalMarkdown, }); } else { // .md-defined agents run with their body as appended system prompt @@ -204,8 +282,31 @@ runRoutes.post("/agent", async (c) => { permissionMode: req.permissionMode, onLog, signal, + harnessExtras: req.harnessExtras ?? req.museReminders, + ignoreLocalMarkdown: req.ignoreLocalMarkdown, }); } + if (req.pilot && result.newSessionId) { + const inTok = result.usage?.inputTokens; + const outTok = result.usage?.outputTokens; + const baseline = + inTok != null + ? estimateTokenBaseline(inTok, req.prompt, { + ignoreLocalMarkdown: req.ignoreLocalMarkdown, + }) + : undefined; + await markPilotSession({ + provider: result.provider, + sessionId: result.newSessionId, + caseId: req.pilotCaseId, + tokensIn: inTok, + tokensOut: outTok, + promptEst: baseline?.promptEst, + baselineEst: baseline?.baselineEst, + baselineNote: baseline?.baselineNote, + ignoreLocalMarkdown: req.ignoreLocalMarkdown, + }).catch(() => undefined); + } const done = { type: "job.done", jobId, inject: result } as const; jobs.finish(jobId, { status: "done", result: done }); bus.publish(done); @@ -234,7 +335,12 @@ runRoutes.post("/session", async (c) => { } const projectDir = req.projectDir || process.cwd(); - const { jobId, signal } = jobs.create("run-session", req.sessionId.slice(0, 16), req.graphId); + const { jobId, signal } = jobs.create( + "run-session", + req.sessionId.slice(0, 16), + req.graphId, + { sessionRef: { provider: req.provider, sessionId: req.sessionId } }, + ); void (async () => { try { bus.publish({ @@ -265,7 +371,9 @@ runRoutes.post("/session", async (c) => { ? await runCopilotAgent(runArgs) : req.provider === "grok" ? await runGrokAgent(runArgs) - : await runClaudeAgent(runArgs); + : req.provider === "muse" + ? await runMuseAgent(runArgs) + : await runClaudeAgent(runArgs); const done = { type: "job.done", jobId, inject: result } as const; jobs.finish(jobId, { status: "done", result: done }); bus.publish(done); diff --git a/packages/server/src/routes/search.ts b/packages/server/src/routes/search.ts index 1186a8d..6225e56 100644 --- a/packages/server/src/routes/search.ts +++ b/packages/server/src/routes/search.ts @@ -11,6 +11,7 @@ const PROVIDERS = new Set([ "codex", "copilot", "grok", + "muse", "payload", ]); const ROLES = new Set(["user", "assistant"]); diff --git a/packages/server/src/routes/sessions.ts b/packages/server/src/routes/sessions.ts index ee76e86..a51a68a 100644 --- a/packages/server/src/routes/sessions.ts +++ b/packages/server/src/routes/sessions.ts @@ -2,6 +2,7 @@ import { Hono } from "hono"; import fs from "node:fs"; import path from "node:path"; import type { SessionRef } from "@threadle/shared"; +import { formatTokCompact } from "@threadle/shared"; import { registry } from "../providers/registry.js"; import { toolInvocationBody, toolSummary } from "../providers/stream.js"; import { renderTranscript } from "../context/extract.js"; @@ -10,13 +11,52 @@ import { buildSessionBundle } from "./bundle.js"; import { FILE_PREVIEW_MAX_BYTES } from "./files.js"; import { globalArtifacts, projectArtifacts } from "./rules.js"; import { readInjects, readPayloadMetas } from "./lineage.js"; -import { threadleConfigDir } from "../paths.js"; +import { threadleConfigDir, sameProjectDir } from "../paths.js"; +import { readPilotSessionMap, type PilotSessionEntry } from "../pilot-sessions/store.js"; +import { ensureBareWorkspace } from "../providers/bare-workspace.js"; import { SESSION_CONTEXT_CONFIG } from "../context/session-context.js"; import { renderInvocationsMarkdown, type InvocationKind, } from "../context/invocations.js"; +async function enrichPilotMeta(ref: SessionRef): Promise { + try { + const [pilotMap, bareDir] = await Promise.all([ + readPilotSessionMap(), + ensureBareWorkspace(), + ]); + return applyPilotEntry(ref, pilotMap.get(`${ref.provider}:${ref.id}`), bareDir); + } catch { + return ref; + } +} + +function applyPilotEntry( + r: SessionRef, + entry: PilotSessionEntry | undefined, + bareDir: string, +): SessionRef { + const marked = Boolean(entry) || sameProjectDir(r.projectDir, bareDir); + if (!marked) return r; + const meta: Record = { ...r.meta, pilot: true }; + if (entry?.caseId) meta.pilotCaseId = entry.caseId; + if (entry?.ignoreLocalMarkdown != null) { + meta.ignoreLocalMarkdown = entry.ignoreLocalMarkdown; + } + if (entry?.baselineEst != null) { + meta.tokenBaselineEst = entry.baselineEst; + meta.tokenPromptEst = entry.promptEst; + meta.tokenBaselineNote = entry.baselineNote; + if (entry.tokensIn != null) meta.pilotTokensIn = entry.tokensIn; + if (entry.tokensOut != null) meta.pilotTokensOut = entry.tokensOut; + meta.tokenBaselineLabel = `~${formatTokCompact(entry.baselineEst)}${ + entry.baselineNote ? ` · ${entry.baselineNote}` : "" + }`; + } + return { ...r, meta }; +} + export const sessionRoutes = new Hono(); // Session ids legitimately contain "/" (composite subagent ids), so the @@ -67,6 +107,20 @@ sessionRoutes.get("/", async (c) => { ); } refs.sort((a, b) => b.updatedAt - a.updatedAt); + + // Tag Settings → test pilot smokes (sidecar mark and/or bare pilot cwd). + try { + const [pilotMap, bareDir] = await Promise.all([ + readPilotSessionMap(), + ensureBareWorkspace(), + ]); + refs = refs.map((r) => + applyPilotEntry(r, pilotMap.get(`${r.provider}:${r.id}`), bareDir), + ); + } catch { + // best-effort — list still works without marks + } + return c.json(refs); }); @@ -178,7 +232,7 @@ sessionRoutes.get("/:provider/detail/:id{.+}", async (c) => { const p = registry.get(c.req.param("provider")); const ref = await p.getSession(c.req.param("id")); if (!ref) return c.json({ error: "session not found" }, 404); - return c.json(ref); + return c.json(await enrichPilotMeta(ref)); }); /** Auto-assembled "blueprint" of everything a session used. */ diff --git a/packages/server/src/routes/settings.ts b/packages/server/src/routes/settings.ts index a154850..ba0807e 100644 --- a/packages/server/src/routes/settings.ts +++ b/packages/server/src/routes/settings.ts @@ -52,7 +52,7 @@ export interface ThreadleSettings { appearance?: "system" | "light" | "dark"; /** * Optional provider accent overrides (`#rrggbb`). Omitted keys keep theme defaults. - * Known ids: claude-code, opencode, cursor, antigravity, codex, copilot, grok. + * Known ids: antigravity, claude-code, codex, copilot, cursor, grok, muse, opencode. */ providerColors?: Partial>; /** When false, canvas MCP client discovery/calls are disabled. Default true. */ @@ -64,6 +64,15 @@ export interface ThreadleSettings { * Empty array (default) = none. `null` = all saved workflows (opt-in). */ mcpPublishAllowlist?: string[] | null; + /** + * Post-answer harness extras (Muse reminders, Claude hooks/slash skills, + * Grok subagents, Antigravity slash skills). When false, inject passes each + * provider's skip switches. Default false — workflows care about wall time. + * Legacy key `museReminders` is still read. + */ + harnessExtras?: boolean; + /** @deprecated use harnessExtras */ + museReminders?: boolean; } const DEFAULTS: ThreadleSettings = { @@ -75,6 +84,7 @@ const DEFAULTS: ThreadleSettings = { mcpClientEnabled: true, mcpDisabledServers: [], mcpPublishAllowlist: [], + harnessExtras: false, }; function settingsFile(): string { @@ -103,13 +113,14 @@ function parsePublishAllowlist(v: unknown): string[] | null { } const PROVIDER_COLOR_IDS = [ - "claude-code", - "opencode", - "cursor", "antigravity", + "claude-code", "codex", "copilot", + "cursor", "grok", + "muse", + "opencode", ] as const; const HEX_RE = /^#([0-9a-fA-F]{3}|[0-9a-fA-F]{6})$/; @@ -176,6 +187,7 @@ export function normalizeSettings(raw: Partial): ThreadleSetti mcpClientEnabled: raw.mcpClientEnabled !== false, mcpDisabledServers: parseStringIds(raw.mcpDisabledServers), mcpPublishAllowlist: parsePublishAllowlist(raw.mcpPublishAllowlist), + harnessExtras: raw.harnessExtras === true || raw.museReminders === true, }; } @@ -191,6 +203,7 @@ export async function readSettings(): Promise { notifications: { ...DEFAULT_NOTIFY }, mcpDisabledServers: [], mcpPublishAllowlist: [], + harnessExtras: false, }; } } @@ -281,6 +294,7 @@ settingsRoutes.put("/", async (c) => { mcpClientEnabled: body.mcpClientEnabled !== false, mcpDisabledServers: parseStringIds(body.mcpDisabledServers), mcpPublishAllowlist: parsePublishAllowlist(body.mcpPublishAllowlist), + harnessExtras: body.harnessExtras === true || body.museReminders === true, }; await fs.promises.mkdir(threadleConfigDir(), { recursive: true }); await fs.promises.writeFile(settingsFile(), JSON.stringify(next, null, 2), "utf8"); diff --git a/packages/server/src/server.ts b/packages/server/src/server.ts index fdf7f29..379a645 100644 --- a/packages/server/src/server.ts +++ b/packages/server/src/server.ts @@ -32,6 +32,7 @@ import { memoryRoutes } from "./routes/memory.js"; import { pluginRoutes } from "./routes/plugins.js"; import { gcTmpFiles } from "./gc.js"; import { appLog, captureConsole, compactJobHistory } from "./jobs.js"; +import { museShare } from "./providers/muse/paths.js"; export interface AppOptions { projectDir: string; @@ -392,6 +393,10 @@ async function storageInfo(providerId: string): Promise : path.join(os.homedir(), ".grok"); const s = await dirSize(p); value = { path: p, ...s }; + } else if (providerId === "muse") { + const p = museShare(); + const s = await dirSize(p); + value = { path: p, ...s }; } } catch { value = undefined; diff --git a/packages/server/src/watch.ts b/packages/server/src/watch.ts index df82344..ebf9115 100644 --- a/packages/server/src/watch.ts +++ b/packages/server/src/watch.ts @@ -10,6 +10,7 @@ import { brainDir as antigravityBrainDir, summariesDbPath } from "./providers/an import { sessionsDir as codexSessionsDir } from "./providers/codex/paths.js"; import { sessionStoreDbPath as copilotSessionStoreDb } from "./providers/copilot/paths.js"; import { sessionsRoot as grokSessionsRoot } from "./providers/grok/paths.js"; +import { sessionsRoot as museSessionsRoot } from "./providers/muse/paths.js"; import { registry } from "./providers/registry.js"; import { opencodeDbPath } from "./providers/opencode/db.js"; @@ -243,6 +244,27 @@ export function startWatchers(): void { }), ); } + + // Muse Code: watch session.jsonl under ~/.local/share/muse/sessions + const museRoot = museSessionsRoot(); + if (fs.existsSync(museRoot)) { + const emitMuse = debounced(() => { + bus.publish({ type: "sessions.changed", provider: "muse" }); + }, 400); + watchers.push( + chokidar + .watch(path.join(museRoot, "**/session.jsonl"), { + ignoreInitial: true, + awaitWriteFinish: { stabilityThreshold: 300, pollInterval: 100 }, + }) + .on("add", emitMuse) + .on("change", emitMuse) + .on("unlink", emitMuse) + .on("error", (err) => { + console.warn(`threadle: muse session watch error: ${String(err)}`); + }), + ); + } } export async function stopWatchers(): Promise { diff --git a/packages/server/src/workflows/executor.ts b/packages/server/src/workflows/executor.ts index e5abb12..e73f8bf 100644 --- a/packages/server/src/workflows/executor.ts +++ b/packages/server/src/workflows/executor.ts @@ -28,6 +28,7 @@ import { clampWaitIdleTimeoutMs, isSessionBusy, WAIT_IDLE_POLL_MS, + nodeHarnessExtrasField, decideWaitIdle, splitIteratorItems, iteratorIsParallel, @@ -55,7 +56,10 @@ import { runAntigravityAgent } from "../providers/antigravity/inject.js"; import { runCodexAgent } from "../providers/codex/inject.js"; import { runCopilotAgent } from "../providers/copilot/inject.js"; import { runGrokAgent } from "../providers/grok/inject.js"; +import { runMuseAgent } from "../providers/muse/inject.js"; import { runOpencodeAgent } from "../providers/opencode/inject.js"; +import { formatDuration, shortId } from "../providers/run-metrics.js"; +import { ensureBareWorkspace } from "../providers/bare-workspace.js"; import { getCustomDef, runCustomDef, LEGACY_PORT } from "../routes/custom-nodes.js"; import { callMcpTool } from "../mcp/client.js"; import { coerceParamsToMcpArgs } from "../mcp/schema.js"; @@ -130,6 +134,31 @@ function fromCached(c: CachedNodeOutput): NodeOutput { }; } +/** Wall-clock ▶/■ around any provider agent call (all providers). */ +async function timedProviderRun( + log: ExecuteOptions["log"], + label: string, + provider: string, + model: string | undefined, + run: () => Promise, +): Promise { + const t0 = Date.now(); + const head = [provider, model].filter(Boolean).join("/"); + log("meta", `▶ ${label}${head ? ` (${head})` : ""}`); + try { + const result = await run(); + const sid = shortId(result.newSessionId); + log( + "meta", + `■ ${label} ${formatDuration(Date.now() - t0)}${sid ? ` · session ${sid}` : ""}`, + ); + return result; + } catch (err) { + log("meta", `■ ${label} failed after ${formatDuration(Date.now() - t0)}`); + throw err; + } +} + /** the value an edge actually carries: a named source port when wired, else the primary text */ function edgeValue(out: NodeOutput, e: GraphEdge): NodeOutput { const port = e.sourceHandle?.startsWith("out:") ? e.sourceHandle.slice(4) : undefined; @@ -700,72 +729,95 @@ export async function executeWorkflow(opts: ExecuteOptions): Promise<{ log("meta", `⋈ merge synthesize via ${node.data.provider}/${node.data.model}`); const onLog = (lane: string, line: string): void => log(lane, `merge: ${line}`); const provider = node.data.provider; + const model = node.data.model; + const agent = node.data.agent; const prompt = text; - let result: InjectResult; - if (provider === "opencode") { - result = await runOpencodeAgent({ - agent: node.data.agent || "build", - model: node.data.model, - prompt, - projectDir: opts.projectDir, - onLog, - signal: opts.signal, - }); - } else if (provider === "cursor") { - result = await runCursorAgent({ - agent: node.data.agent || "agent", - model: node.data.model, - prompt, - projectDir: opts.projectDir, - onLog, - signal: opts.signal, - }); - } else if (provider === "antigravity") { - result = await runAntigravityAgent({ - agent: node.data.agent || "agent", - model: node.data.model, - prompt, - projectDir: opts.projectDir, - onLog, - signal: opts.signal, - }); - } else if (provider === "codex") { - result = await runCodexAgent({ - agent: node.data.agent || "codex", - model: node.data.model, - prompt, - projectDir: opts.projectDir, - onLog, - signal: opts.signal, - }); - } else if (provider === "copilot") { - result = await runCopilotAgent({ - agent: node.data.agent || "copilot", - model: node.data.model, - prompt, - projectDir: opts.projectDir, - onLog, - signal: opts.signal, - }); - } else if (provider === "grok") { - result = await runGrokAgent({ - agent: node.data.agent || "grok", - model: node.data.model, - prompt, - projectDir: opts.projectDir, - onLog, - signal: opts.signal, - }); - } else { - result = await runClaudeAgent({ - agent: node.data.agent, - model: node.data.model, - prompt, - projectDir: opts.projectDir, - onLog, - signal: opts.signal, - }); - } + const result = await timedProviderRun( + log, + "merge", + provider, + model, + async () => { + if (provider === "opencode") { + return runOpencodeAgent({ + agent: agent || "build", + model, + prompt, + projectDir: opts.projectDir, + onLog, + signal: opts.signal, + }); + } + if (provider === "cursor") { + return runCursorAgent({ + agent: agent || "agent", + model, + prompt, + projectDir: opts.projectDir, + onLog, + signal: opts.signal, + }); + } + if (provider === "antigravity") { + return runAntigravityAgent({ + agent: agent || "agent", + model, + prompt, + projectDir: opts.projectDir, + onLog, + signal: opts.signal, + }); + } + if (provider === "codex") { + return runCodexAgent({ + agent: agent || "codex", + model, + prompt, + projectDir: opts.projectDir, + onLog, + signal: opts.signal, + }); + } + if (provider === "copilot") { + return runCopilotAgent({ + agent: agent || "copilot", + model, + prompt, + projectDir: opts.projectDir, + onLog, + signal: opts.signal, + }); + } + if (provider === "grok") { + return runGrokAgent({ + agent: agent || "grok", + model, + prompt, + projectDir: opts.projectDir, + onLog, + signal: opts.signal, + }); + } + if (provider === "muse") { + return runMuseAgent({ + agent: agent || "muse", + model, + prompt, + projectDir: opts.projectDir, + onLog, + signal: opts.signal, + }); + } + return runClaudeAgent({ + agent, + model, + prompt, + projectDir: opts.projectDir, + onLog, + signal: opts.signal, + }); + }, + ); text = result.resultText ?? (await lastAssistantText(result.provider, result.newSessionId)) ?? @@ -1141,10 +1193,17 @@ export async function executeWorkflow(opts: ExecuteOptions): Promise<{ }); const linkedNode = linkedEdge ? nodeById(linkedEdge.target) : undefined; const linkedData = linkedNode?.data.type === "session" ? linkedNode.data : undefined; - const projectDir = await resolveNodeProjectDir( + const resolvedProjectDir = await resolveNodeProjectDir( linkedData?.snapshot?.projectDir, `agent "${label(node)}"`, ); + const ignoreLocalMarkdown = data.ignoreLocalMarkdown === true; + const projectDir = ignoreLocalMarkdown + ? await ensureBareWorkspace() + : resolvedProjectDir; + if (ignoreLocalMarkdown) { + log("meta", `${data.ref.name}: bare workspace · ignore local md`); + } const collected: string[] = new Array(prompts.length); let last: InjectResult | undefined; const provider = data.ref.provider; @@ -1163,84 +1222,116 @@ export async function executeWorkflow(opts: ExecuteOptions): Promise<{ const onLog = (lane: string, line: string): void => log(lane, `${data.ref.name}: ${line}`); const prompt = prompts[i]!; - let result: InjectResult; - if (provider === "opencode") { - result = await runOpencodeAgent({ - agent: data.ref.name, - model: data.model, - prompt, - projectDir, - sessionId, - onLog, - signal: opts.signal, - }); - } else if (provider === "cursor") { - result = await runCursorAgent({ - agent: data.ref.name, - model: data.model, - prompt, - projectDir, - sessionId, - onLog, - signal: opts.signal, - }); - } else if (provider === "antigravity") { - result = await runAntigravityAgent({ - agent: data.ref.name, - model: data.model, - prompt, - projectDir, - sessionId, - onLog, - signal: opts.signal, - }); - } else if (provider === "codex") { - result = await runCodexAgent({ - agent: data.ref.name, - model: data.model, - prompt, - projectDir, - sessionId, - sandbox: data.sandbox, - askForApproval: data.askForApproval, - onLog, - signal: opts.signal, - }); - } else if (provider === "copilot") { - result = await runCopilotAgent({ - agent: data.ref.name, - model: data.model, - prompt, - projectDir, - sessionId, - onLog, - signal: opts.signal, - }); - } else if (provider === "grok") { - result = await runGrokAgent({ - agent: data.ref.name, - model: data.model, - prompt, - projectDir, - sessionId, - onLog, - signal: opts.signal, - }); - } else { - const defs = await registry.get("claude-code").listAgents({ projectDir }); - const def = defs.find((a) => a.name === data.ref.name); - result = await runClaudeAgent({ - agent: data.ref.name, - agentSystemPrompt: def?.raw ? matter(def.raw).content.trim() : undefined, - model: data.model, - prompt, - projectDir, - sessionId, - permissionMode: data.permissionMode, - onLog, - signal: opts.signal, - }); - } + const harnessExtras = nodeHarnessExtrasField(data); + const result = await timedProviderRun( + log, + data.ref.name, + provider, + data.model, + async () => { + if (provider === "opencode") { + return runOpencodeAgent({ + agent: data.ref.name, + model: data.model, + prompt, + projectDir, + sessionId, + onLog, + signal: opts.signal, + }); + } + if (provider === "cursor") { + return runCursorAgent({ + agent: data.ref.name, + model: data.model, + prompt, + projectDir, + sessionId, + onLog, + signal: opts.signal, + ignoreLocalMarkdown, + }); + } + if (provider === "antigravity") { + return runAntigravityAgent({ + agent: data.ref.name, + model: data.model, + prompt, + projectDir, + sessionId, + onLog, + signal: opts.signal, + harnessExtras, + }); + } + if (provider === "codex") { + return runCodexAgent({ + agent: data.ref.name, + model: data.model, + prompt, + projectDir, + sessionId, + sandbox: data.sandbox, + askForApproval: data.askForApproval, + onLog, + signal: opts.signal, + ignoreLocalMarkdown, + }); + } + if (provider === "copilot") { + return runCopilotAgent({ + agent: data.ref.name, + model: data.model, + prompt, + projectDir, + sessionId, + onLog, + signal: opts.signal, + ignoreLocalMarkdown, + }); + } + if (provider === "grok") { + return runGrokAgent({ + agent: data.ref.name, + model: data.model, + prompt, + projectDir, + sessionId, + onLog, + signal: opts.signal, + harnessExtras, + }); + } + if (provider === "muse") { + return runMuseAgent({ + agent: data.ref.name, + model: data.model, + prompt, + projectDir, + sessionId, + onLog, + signal: opts.signal, + harnessExtras, + ignoreLocalMarkdown, + }); + } + const defs = await registry.get("claude-code").listAgents({ projectDir }); + const def = defs.find((a) => a.name === data.ref.name); + return runClaudeAgent({ + agent: data.ref.name, + agentSystemPrompt: def?.raw ? matter(def.raw).content.trim() : undefined, + model: data.model, + prompt, + projectDir, + sessionId, + permissionMode: data.permissionMode, + onLog, + signal: opts.signal, + harnessExtras, + ignoreLocalMarkdown, + }); + }, + ); const text = result.resultText ?? (await lastAssistantText(result.provider, result.newSessionId)); if (text) collected[i] = text; @@ -1307,20 +1398,28 @@ export async function executeWorkflow(opts: ExecuteOptions): Promise<{ onLog, signal: opts.signal, }; - const result = - ref.provider === "opencode" - ? await runOpencodeAgent(sessionArgs) - : ref.provider === "cursor" - ? await runCursorAgent(sessionArgs) - : ref.provider === "antigravity" - ? await runAntigravityAgent(sessionArgs) - : ref.provider === "codex" - ? await runCodexAgent(sessionArgs) - : ref.provider === "copilot" - ? await runCopilotAgent(sessionArgs) - : ref.provider === "grok" - ? await runGrokAgent(sessionArgs) - : await runClaudeAgent(sessionArgs); + const result = await timedProviderRun( + log, + `session ${shortId(ref.sessionId) || ref.provider}`, + ref.provider, + undefined, + async () => + ref.provider === "opencode" + ? runOpencodeAgent(sessionArgs) + : ref.provider === "cursor" + ? runCursorAgent(sessionArgs) + : ref.provider === "antigravity" + ? runAntigravityAgent(sessionArgs) + : ref.provider === "codex" + ? runCodexAgent(sessionArgs) + : ref.provider === "copilot" + ? runCopilotAgent(sessionArgs) + : ref.provider === "grok" + ? runGrokAgent(sessionArgs) + : ref.provider === "muse" + ? runMuseAgent(sessionArgs) + : runClaudeAgent(sessionArgs), + ); outputs.set(node.id, { text: result.resultText ?? (await lastAssistantText(result.provider, result.newSessionId)), session: { provider: result.provider, sessionId: result.newSessionId }, diff --git a/packages/server/src/workflows/provider-agent.ts b/packages/server/src/workflows/provider-agent.ts index 358954a..06390a3 100644 --- a/packages/server/src/workflows/provider-agent.ts +++ b/packages/server/src/workflows/provider-agent.ts @@ -5,6 +5,7 @@ import { runAntigravityAgent } from "../providers/antigravity/inject.js"; import { runCodexAgent } from "../providers/codex/inject.js"; import { runCopilotAgent } from "../providers/copilot/inject.js"; import { runGrokAgent } from "../providers/grok/inject.js"; +import { runMuseAgent } from "../providers/muse/inject.js"; import { runOpencodeAgent } from "../providers/opencode/inject.js"; import type { LogSink } from "../providers/stream.js"; @@ -39,6 +40,8 @@ export async function runProviderAgent(opts: ProviderAgentOpts): Promise { }); it("codex: safetyArgs defaults and overrides", () => { - expect(safetyArgs()).toEqual([ - "--sandbox", - "workspace-write", - "--ask-for-approval", - "never", - ]); + expect(safetyArgs()).toEqual(["--approve-for-me"]); expect( safetyArgs({ sandbox: "read-only", askForApproval: "on-request" }), - ).toEqual([ - "--sandbox", - "read-only", - "--ask-for-approval", - "on-request", - ]); + ).toEqual(["--sandbox", "read-only"]); + expect( + safetyArgs({ sandbox: "workspace-write", askForApproval: "on-request" }), + ).toEqual(["--sandbox", "workspace-write"]); + expect( + safetyArgs({ sandbox: "read-only", askForApproval: "never" }), + ).toEqual(["--sandbox", "read-only"]); }); it("codex: execArgs includes sandbox from opts", () => { @@ -48,6 +44,7 @@ describe("agent permission / sandbox argv", () => { }); expect(args).toContain("--sandbox"); expect(args[args.indexOf("--sandbox") + 1]).toBe("danger-full-access"); - expect(args[args.indexOf("--ask-for-approval") + 1]).toBe("on-failure"); + expect(args).not.toContain("--approve-for-me"); + expect(args).not.toContain("--ask-for-approval"); }); }); diff --git a/packages/server/test/agent-security-argv.test.ts b/packages/server/test/agent-security-argv.test.ts index 9bba40f..16be833 100644 --- a/packages/server/test/agent-security-argv.test.ts +++ b/packages/server/test/agent-security-argv.test.ts @@ -46,19 +46,14 @@ describe("claude agent argv", () => { }); describe("codex safety argv", () => { - it("defaults sandbox and ask-for-approval", () => { - expect(safetyArgs()).toEqual([ - "--sandbox", - "workspace-write", - "--ask-for-approval", - "never", - ]); + it("defaults to --approve-for-me (workspace-write implied)", () => { + expect(safetyArgs()).toEqual(["--approve-for-me"]); }); - it("uses opts.sandbox / askForApproval when set", () => { + it("uses explicit --sandbox when not workspace-write", () => { expect( safetyArgs({ sandbox: "read-only", askForApproval: "on-request" }), - ).toEqual(["--sandbox", "read-only", "--ask-for-approval", "on-request"]); + ).toEqual(["--sandbox", "read-only"]); }); it("threads safety into execArgs", () => { @@ -73,11 +68,18 @@ describe("codex safety argv", () => { "--json", "--sandbox", "danger-full-access", - "--ask-for-approval", - "never", "--model", "gpt-5", "do it", ]); }); + + it("default execArgs uses approve-for-me", () => { + expect(execArgs({ prompt: "hi" })).toEqual([ + "exec", + "--json", + "--approve-for-me", + "hi", + ]); + }); }); diff --git a/packages/server/test/check.test.ts b/packages/server/test/check.test.ts index d9bd070..328bca2 100644 --- a/packages/server/test/check.test.ts +++ b/packages/server/test/check.test.ts @@ -19,6 +19,7 @@ vi.mock("../src/providers/registry.js", () => ({ { id: "codex", available: false }, { id: "copilot", available: false }, { id: "grok", available: false }, + { id: "muse", available: false }, ], }, })); @@ -31,6 +32,7 @@ const PROVIDER_IDS = [ "codex", "copilot", "grok", + "muse", ] as const; const CLI_IDS = [ @@ -41,6 +43,7 @@ const CLI_IDS = [ "cli:codex", "cli:copilot", "cli:grok", + "cli:muse", ] as const; /** `runCheck` treats baked UI as critical; unit CI may not have built yet. */ @@ -104,6 +107,7 @@ describe("threadle check", () => { "fixture:antigravity", "fixture:opencode", "fixture:grok", + "fixture:muse", ]; for (const id of fixtureIds) { expect(byId[id], `missing ${id}`).toBeDefined(); @@ -123,6 +127,7 @@ describe("golden fixture probe", () => { const root = goldenFixturesRoot(); expect(root).toBeTruthy(); expect(fs.existsSync(path.join(root!, "grok"))).toBe(true); + expect(fs.existsSync(path.join(root!, "muse"))).toBe(true); }); it("probes all provider fixtures without throwing", async () => { @@ -135,6 +140,7 @@ describe("golden fixture probe", () => { "fixture:codex", "fixture:cursor", "fixture:grok", + "fixture:muse", "fixture:opencode", ].sort(), ); diff --git a/packages/server/test/cli-inventory.test.ts b/packages/server/test/cli-inventory.test.ts index 9e9ccc7..61c07a5 100644 --- a/packages/server/test/cli-inventory.test.ts +++ b/packages/server/test/cli-inventory.test.ts @@ -31,6 +31,10 @@ vi.mock("../src/providers/grok/inject.js", () => ({ runGrokAgent: vi.fn(), listGrokModels: async () => ["grok-mock"], })); +vi.mock("../src/providers/muse/inject.js", () => ({ + runMuseAgent: vi.fn(), + listMuseModels: async () => ["muse-mock"], +})); vi.mock("../src/providers/claude-code/inject.js", () => ({ runClaudeAgent: vi.fn(), })); diff --git a/packages/server/test/codex-event-to-logs.test.ts b/packages/server/test/codex-event-to-logs.test.ts new file mode 100644 index 0000000..68a1246 --- /dev/null +++ b/packages/server/test/codex-event-to-logs.test.ts @@ -0,0 +1,54 @@ +import { describe, expect, it } from "vitest"; +import { eventToLogs } from "../src/providers/codex/inject.js"; + +describe("codex eventToLogs", () => { + it("maps agent_message and text", () => { + expect(eventToLogs(JSON.stringify({ type: "agent_message", message: "hi" }))).toEqual([ + ["text", "hi"], + ]); + expect(eventToLogs(JSON.stringify({ text: "plain" }))).toEqual([["text", "plain"]]); + }); + + it("maps reasoning to thinking", () => { + expect( + eventToLogs( + JSON.stringify({ + type: "reasoning", + summary: [{ type: "summary_text", text: "consider options" }], + }), + ), + ).toEqual([["thinking", "consider options"]]); + + expect( + eventToLogs( + JSON.stringify({ + payload: { type: "reasoning", content: "deep thought" }, + }), + ), + ).toEqual([["thinking", "deep thought"]]); + }); + + it("maps function_call / tool_call to tool", () => { + expect( + eventToLogs( + JSON.stringify({ + type: "function_call", + name: "shell", + arguments: JSON.stringify({ command: "ls -la" }), + }), + ), + ).toEqual([["tool", "shell: ls -la"]]); + + expect( + eventToLogs( + JSON.stringify({ + item: { type: "tool_call", name: "Read", input: { path: "/tmp/a.ts" } }, + }), + ), + ).toEqual([["tool", "Read: /tmp/a.ts"]]); + }); + + it("ignores garbage", () => { + expect(eventToLogs("not-json")).toEqual([]); + }); +}); diff --git a/packages/server/test/codex-provider.test.ts b/packages/server/test/codex-provider.test.ts index fa450cc..92e6191 100644 --- a/packages/server/test/codex-provider.test.ts +++ b/packages/server/test/codex-provider.test.ts @@ -76,6 +76,19 @@ const rolloutLines = [ type: "unknown_future", payload: { ignore: true }, }, + { + timestamp: "2026-09-15T21:35:33.000Z", + type: "token_usage_record", + payload: { + session_id: sessionId, + usage: { + input_tokens: 100, + output_tokens: 12, + reasoning_output_tokens: 0, + total_tokens: 112, + }, + }, + }, ]; function writeFixture(): void { @@ -185,9 +198,39 @@ describe("codex discover", () => { expect(hit!.ref.title).toBe("Fixture Codex"); expect(hit!.ref.projectDir).toBe(projectDir); expect(hit!.ref.model).toBe("gpt-5"); - expect(hit!.ref.tokensIn).toBeGreaterThan(0); - expect(hit!.ref.tokensOut).toBeGreaterThan(0); - expect(hit!.ref.meta?.tokenSource).toBe("estimate"); + expect(hit!.ref.tokensIn).toBe(100); + expect(hit!.ref.tokensOut).toBe(12); + expect(hit!.ref.meta?.tokenSource).toBe("cli"); + }); + + it("parses token_usage_record and token_count events", async () => { + const { parseCodexUsageFromEvent, findCodexUsageInJsonl } = await import( + "../src/providers/codex/usage.js" + ); + const fromRecord = parseCodexUsageFromEvent({ + type: "token_usage_record", + payload: { usage: { input_tokens: 50, output_tokens: 3 } }, + }); + expect(fromRecord).toEqual({ + tokensIn: 50, + tokensOut: 3, + tokensReasoning: undefined, + source: "cli", + }); + const fromCount = findCodexUsageInJsonl( + JSON.stringify({ + type: "event_msg", + payload: { + type: "token_count", + info: { + last_token_usage: { input_tokens: 80, output_tokens: 4 }, + }, + }, + }), + ); + expect(fromCount?.tokensIn).toBe(80); + expect(fromCount?.tokensOut).toBe(4); + expect(fromCount?.source).toBe("cli"); }); it("estimates usage from transcript chars", async () => { diff --git a/packages/server/test/copilot-provider.test.ts b/packages/server/test/copilot-provider.test.ts index a1a61b6..3dc83b6 100644 --- a/packages/server/test/copilot-provider.test.ts +++ b/packages/server/test/copilot-provider.test.ts @@ -147,4 +147,19 @@ describe("copilot provider", () => { expect(sess?.tokensReasoning).toBe(10); expect(sess?.model).toBe("gpt-5"); }); + + it("parses --usage-output-file JSON shapes", async () => { + const { parseCopilotUsageFile } = await import("../src/providers/copilot/usage.js"); + expect( + parseCopilotUsageFile( + JSON.stringify({ input_tokens: 10, output_tokens: 2 }), + ), + ).toEqual({ inputTokens: 10, outputTokens: 2 }); + expect( + parseCopilotUsageFile( + JSON.stringify({ usage: { inputTokens: 7, outputTokens: 1 } }), + ), + ).toEqual({ inputTokens: 7, outputTokens: 1 }); + expect(parseCopilotUsageFile("{}")).toBeUndefined(); + }); }); diff --git a/packages/server/test/fixtures/providers/README.md b/packages/server/test/fixtures/providers/README.md index a34237d..efd3d0a 100644 --- a/packages/server/test/fixtures/providers/README.md +++ b/packages/server/test/fixtures/providers/README.md @@ -5,3 +5,8 @@ # grok/ — full `~/.grok` home layout (set GROK_HOME to this folder in tests): # sessions/project//{summary,chat_history,updates,usage,plan*} (+ .cwd → /tmp/grok-fx) # Three sessions: plan-mode, no-plan, updates-only (no chat_history). +# +# muse/ — `MUSE_DATA_DIR` share layout: +# sessions//session.jsonl (MSP; unknown payload types must skip) +# sessions//subagent//session.jsonl (parent→child lineage) +# exec/pong.jsonl — sample `muse exec --json` stdout (task lifecycle + deltas) diff --git a/packages/server/test/fixtures/providers/codex/rollout.jsonl b/packages/server/test/fixtures/providers/codex/rollout.jsonl index 45556a7..4da0bc7 100644 --- a/packages/server/test/fixtures/providers/codex/rollout.jsonl +++ b/packages/server/test/fixtures/providers/codex/rollout.jsonl @@ -2,3 +2,4 @@ {"timestamp":"2026-09-15T21:35:28.000Z","type":"response_item","payload":{"type":"message","role":"user","content":[{"type":"input_text","text":"hello codex fixture"}]}} {"timestamp":"2026-09-15T21:35:29.000Z","type":"response_item","payload":{"type":"message","role":"assistant","content":[{"type":"output_text","text":"hi from codex"}]}} {"timestamp":"2026-09-15T21:35:30.000Z","type":"unknown_future","payload":{"ignore":true}} +{"timestamp":"2026-09-15T21:35:31.000Z","type":"token_usage_record","payload":{"session_id":"aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee","usage":{"input_tokens":100,"output_tokens":12,"reasoning_output_tokens":0,"total_tokens":112}}} diff --git a/packages/server/test/fixtures/providers/muse/exec/pong.jsonl b/packages/server/test/fixtures/providers/muse/exec/pong.jsonl new file mode 100644 index 0000000..b3b3fa0 --- /dev/null +++ b/packages/server/test/fixtures/providers/muse/exec/pong.jsonl @@ -0,0 +1,32 @@ +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c350","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":2,"recorded_at":1790171086752059,"record_type":"reconciliation","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"runtime.command.accepted","payload_schema_version":1,"payload":{"kind":"command_accepted","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","client_id":null,"command_kind":"turn.submit"}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c351","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":3,"recorded_at":1790171086752060,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"session.run.linked","payload_schema_version":1,"payload":{"kind":"session_run_linked","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"}}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c352","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":4,"recorded_at":1790171086752061,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"run.model.configured","payload_schema_version":1,"payload":{"kind":"run_model_configured","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"provider_id":"meta","profile_id":null,"model_id":"muse-spark-1.3-contributor","display_label":"muse-spark-1.3-contributor","source":"startup"}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c354","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":5,"recorded_at":1790171086752063,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"session.workspace_branch.observed","payload_schema_version":1,"payload":{"kind":"workspace_branch_observed","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","record":{"command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","workspace_root":"/Users/zfarbp/Development/threadle","reference":{"kind":"branch","name":"main"},"vcs":"git","commit":"9c733e8e5b83"}}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c355","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":6,"recorded_at":1790171086752064,"record_type":"status","durability":"ephemeral","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"turn.input.user","payload_schema_version":1,"payload":{"kind":"turn_input_user","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"prompt":"Reply with exactly: pong\n"}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c357","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":7,"recorded_at":1790171086752066,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"run.lifecycle.started","payload_schema_version":1,"payload":{"kind":"run_started","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"prompt":"Reply with exactly: pong\n"}} +{"schema_version":1,"id":"e7d1608f-60e6-57eb-af60-a2deb0da25d3","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":8,"recorded_at":1790171086752068,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"task.stream.linked","payload_schema_version":1,"payload":{"kind":"task_stream_linked","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"task_id":"c2ba7608-960b-5c16-9f4f-95e3a9cd18c0","task_stream":{"kind":"task","id":"c2ba7608-960b-5c16-9f4f-95e3a9cd18c0"}}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c359","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":10,"recorded_at":1790171086777976,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"task.stream.linked","payload_schema_version":1,"payload":{"kind":"task_stream_linked","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"task_id":"01a0ce83-026b-7e10-b911-4517997e7887","task_stream":{"kind":"task","id":"01a0ce83-026b-7e10-b911-4517997e7887"}}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c35b","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":11,"recorded_at":1790171086777978,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"task.lifecycle.proposed","payload_schema_version":1,"payload":{"kind":"task_lifecycle","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"task_stream":{"kind":"task","id":"01a0ce83-026b-7e10-b911-4517997e7887"},"task_id":"01a0ce83-026b-7e10-b911-4517997e7887","event":{"kind":"proposed","task_id":"01a0ce83-026b-7e10-b911-4517997e7887","task_kind":"reminder.agent.skill-reminder"}}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c35d","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":12,"recorded_at":1790171086777980,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"task.lifecycle.accepted","payload_schema_version":1,"payload":{"kind":"task_lifecycle","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"task_stream":{"kind":"task","id":"01a0ce83-026b-7e10-b911-4517997e7887"},"task_id":"01a0ce83-026b-7e10-b911-4517997e7887","event":{"kind":"accepted","task_id":"01a0ce83-026b-7e10-b911-4517997e7887"}}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c35f","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":13,"recorded_at":1790171086777982,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"task.lifecycle.scheduled","payload_schema_version":1,"payload":{"kind":"task_lifecycle","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"task_stream":{"kind":"task","id":"01a0ce83-026b-7e10-b911-4517997e7887"},"task_id":"01a0ce83-026b-7e10-b911-4517997e7887","event":{"kind":"scheduled","task_id":"01a0ce83-026b-7e10-b911-4517997e7887","idempotency_key":"reminder_child:skill-reminder:1"}}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c361","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":14,"recorded_at":1790171086777984,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"task.lifecycle.side_effect_intent","payload_schema_version":1,"payload":{"kind":"task_lifecycle","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"task_stream":{"kind":"task","id":"01a0ce83-026b-7e10-b911-4517997e7887"},"task_id":"01a0ce83-026b-7e10-b911-4517997e7887","event":{"kind":"side_effect_intent","task_id":"01a0ce83-026b-7e10-b911-4517997e7887","operation":"reminder.child_run","idempotency_key":"reminder_child:skill-reminder:1","policy_decision":"reminder_child:read_only:subagent_tool_auto_approval","cancellation_handle":null,"parent_task_id":null}}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c363","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":15,"recorded_at":1790171086777986,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"task.lifecycle.started","payload_schema_version":1,"payload":{"kind":"task_lifecycle","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"task_stream":{"kind":"task","id":"01a0ce83-026b-7e10-b911-4517997e7887"},"task_id":"01a0ce83-026b-7e10-b911-4517997e7887","event":{"kind":"started","task_id":"01a0ce83-026b-7e10-b911-4517997e7887"}}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c365","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":16,"recorded_at":1790171086777988,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"task.stream.linked","payload_schema_version":1,"payload":{"kind":"task_stream_linked","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"task_id":"01a0ce83-0293-7eb1-88bb-7fd481c35a3d","task_stream":{"kind":"task","id":"01a0ce83-0293-7eb1-88bb-7fd481c35a3d"}}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c367","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":17,"recorded_at":1790171086777990,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"task.lifecycle.proposed","payload_schema_version":1,"payload":{"kind":"task_lifecycle","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"task_stream":{"kind":"task","id":"01a0ce83-0293-7eb1-88bb-7fd481c35a3d"},"task_id":"01a0ce83-0293-7eb1-88bb-7fd481c35a3d","event":{"kind":"proposed","task_id":"01a0ce83-0293-7eb1-88bb-7fd481c35a3d","task_kind":"model.meta.response"}}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c369","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":18,"recorded_at":1790171086777992,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"task.lifecycle.accepted","payload_schema_version":1,"payload":{"kind":"task_lifecycle","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"task_stream":{"kind":"task","id":"01a0ce83-0293-7eb1-88bb-7fd481c35a3d"},"task_id":"01a0ce83-0293-7eb1-88bb-7fd481c35a3d","event":{"kind":"accepted","task_id":"01a0ce83-0293-7eb1-88bb-7fd481c35a3d"}}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c36b","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":19,"recorded_at":1790171086777994,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"task.lifecycle.scheduled","payload_schema_version":1,"payload":{"kind":"task_lifecycle","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"task_stream":{"kind":"task","id":"01a0ce83-0293-7eb1-88bb-7fd481c35a3d"},"task_id":"01a0ce83-0293-7eb1-88bb-7fd481c35a3d","event":{"kind":"scheduled","task_id":"01a0ce83-0293-7eb1-88bb-7fd481c35a3d","idempotency_key":"model:de066adf-c6de-4dbf-8afb-daa8b36b18c2:01a0ce83-0293-7eb1-88bb-7fd481c35a3d"}}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c36d","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":20,"recorded_at":1790171086777996,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"task.lifecycle.side_effect_intent","payload_schema_version":1,"payload":{"kind":"task_lifecycle","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"task_stream":{"kind":"task","id":"01a0ce83-0293-7eb1-88bb-7fd481c35a3d"},"task_id":"01a0ce83-0293-7eb1-88bb-7fd481c35a3d","event":{"kind":"side_effect_intent","task_id":"01a0ce83-0293-7eb1-88bb-7fd481c35a3d","operation":"model.meta.response","idempotency_key":"model:de066adf-c6de-4dbf-8afb-daa8b36b18c2:01a0ce83-0293-7eb1-88bb-7fd481c35a3d","policy_decision":"not_applicable","cancellation_handle":null,"parent_task_id":null}}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c36f","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":21,"recorded_at":1790171086777998,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"task.lifecycle.started","payload_schema_version":1,"payload":{"kind":"task_lifecycle","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"task_stream":{"kind":"task","id":"01a0ce83-0293-7eb1-88bb-7fd481c35a3d"},"task_id":"01a0ce83-0293-7eb1-88bb-7fd481c35a3d","event":{"kind":"started","task_id":"01a0ce83-0293-7eb1-88bb-7fd481c35a3d"}}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c371","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":22,"recorded_at":1790171086778000,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"task.lifecycle.status","payload_schema_version":1,"payload":{"kind":"task_lifecycle","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"task_stream":{"kind":"task","id":"01a0ce83-0293-7eb1-88bb-7fd481c35a3d"},"task_id":"01a0ce83-0293-7eb1-88bb-7fd481c35a3d","event":{"kind":"status","task_id":"01a0ce83-0293-7eb1-88bb-7fd481c35a3d","message":"opening meta model stream attempt 1/10","details":{"phase":"opening_stream","facets":[{"kind":"external_attempt","system":"meta","operation":"model.response","attempt":1,"max_attempts":10},{"kind":"producer","detail":{"kind":"provider","provider":"meta"}}]}}}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c373","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":23,"recorded_at":1790171086778002,"record_type":"status","durability":"ephemeral","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"run.output.delta","payload_schema_version":1,"payload":{"kind":"run_output_delta","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"text":"pong"}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c375","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":24,"recorded_at":1790171086778004,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"task.lifecycle.status","payload_schema_version":1,"payload":{"kind":"task_lifecycle","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"task_stream":{"kind":"task","id":"01a0ce83-0293-7eb1-88bb-7fd481c35a3d"},"task_id":"01a0ce83-0293-7eb1-88bb-7fd481c35a3d","event":{"kind":"status","task_id":"01a0ce83-0293-7eb1-88bb-7fd481c35a3d","message":"completed meta model stream attempt 1/10","details":{"phase":"stream_succeeded","facets":[{"kind":"external_attempt","system":"meta","operation":"model.response","attempt":1,"max_attempts":10,"error_kind":"stream_succeeded"},{"kind":"producer","detail":{"kind":"provider","provider":"meta","request_id":"c9e330c0-28e0-43b2-a44f-8931f2ec03a5","response_id":"resp_6ab3d7d288c861238ac24284","model":"muse-spark-1.3-contributor","stream":{"bytes_read":208542,"wire_events_seen":10,"last_wire_event_type":"response.completed","eof":false,"eof_buffer_len":0,"time_to_first_event_ms":175}}}]}}}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c377","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":25,"recorded_at":1790171086778006,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"task.lifecycle.completed","payload_schema_version":1,"payload":{"kind":"task_lifecycle","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"task_stream":{"kind":"task","id":"01a0ce83-0293-7eb1-88bb-7fd481c35a3d"},"task_id":"01a0ce83-0293-7eb1-88bb-7fd481c35a3d","event":{"kind":"completed","task_id":"01a0ce83-0293-7eb1-88bb-7fd481c35a3d"}}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c379","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":26,"recorded_at":1790171086778008,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"task.stream.linked","payload_schema_version":1,"payload":{"kind":"task_stream_linked","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"task_id":"01a0ce83-1063-7353-97b9-ae66256d2de6","task_stream":{"kind":"task","id":"01a0ce83-1063-7353-97b9-ae66256d2de6"}}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c37b","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":27,"recorded_at":1790171086778010,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"task.lifecycle.proposed","payload_schema_version":1,"payload":{"kind":"task_lifecycle","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"task_stream":{"kind":"task","id":"01a0ce83-1063-7353-97b9-ae66256d2de6"},"task_id":"01a0ce83-1063-7353-97b9-ae66256d2de6","event":{"kind":"proposed","task_id":"01a0ce83-1063-7353-97b9-ae66256d2de6","task_kind":"reminder.agent.verify-reminder"}}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c37d","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":28,"recorded_at":1790171086778012,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"task.lifecycle.accepted","payload_schema_version":1,"payload":{"kind":"task_lifecycle","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"task_stream":{"kind":"task","id":"01a0ce83-1063-7353-97b9-ae66256d2de6"},"task_id":"01a0ce83-1063-7353-97b9-ae66256d2de6","event":{"kind":"accepted","task_id":"01a0ce83-1063-7353-97b9-ae66256d2de6"}}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c37f","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":29,"recorded_at":1790171086778014,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"task.lifecycle.scheduled","payload_schema_version":1,"payload":{"kind":"task_lifecycle","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"task_stream":{"kind":"task","id":"01a0ce83-1063-7353-97b9-ae66256d2de6"},"task_id":"01a0ce83-1063-7353-97b9-ae66256d2de6","event":{"kind":"scheduled","task_id":"01a0ce83-1063-7353-97b9-ae66256d2de6","idempotency_key":"reminder_child:verify-reminder:1"}}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c381","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":30,"recorded_at":1790171086778016,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"task.lifecycle.side_effect_intent","payload_schema_version":1,"payload":{"kind":"task_lifecycle","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"task_stream":{"kind":"task","id":"01a0ce83-1063-7353-97b9-ae66256d2de6"},"task_id":"01a0ce83-1063-7353-97b9-ae66256d2de6","event":{"kind":"side_effect_intent","task_id":"01a0ce83-1063-7353-97b9-ae66256d2de6","operation":"reminder.child_run","idempotency_key":"reminder_child:verify-reminder:1","policy_decision":"reminder_child:read_only:subagent_tool_auto_approval","cancellation_handle":null,"parent_task_id":null}}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c383","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":31,"recorded_at":1790171086778018,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"task.lifecycle.started","payload_schema_version":1,"payload":{"kind":"task_lifecycle","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"task_stream":{"kind":"task","id":"01a0ce83-1063-7353-97b9-ae66256d2de6"},"task_id":"01a0ce83-1063-7353-97b9-ae66256d2de6","event":{"kind":"started","task_id":"01a0ce83-1063-7353-97b9-ae66256d2de6"}}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c385","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":32,"recorded_at":1790171086778020,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"task.lifecycle.completed","payload_schema_version":1,"payload":{"kind":"task_lifecycle","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"task_stream":{"kind":"task","id":"01a0ce83-026b-7e10-b911-4517997e7887"},"task_id":"01a0ce83-026b-7e10-b911-4517997e7887","event":{"kind":"completed","task_id":"01a0ce83-026b-7e10-b911-4517997e7887"}}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c387","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":33,"recorded_at":1790171086778022,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"task.lifecycle.completed","payload_schema_version":1,"payload":{"kind":"task_lifecycle","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"task_stream":{"kind":"task","id":"01a0ce83-1063-7353-97b9-ae66256d2de6"},"task_id":"01a0ce83-1063-7353-97b9-ae66256d2de6","event":{"kind":"completed","task_id":"01a0ce83-1063-7353-97b9-ae66256d2de6"}}} +{"schema_version":1,"id":"018f0000-0000-7000-8000-00000000c389","stream":{"kind":"session","id":"de4c62fb-7579-4e49-9c7b-a842169a076e"},"sequence":34,"recorded_at":1790171086778024,"record_type":"event","durability":"durable","causation_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","payload_type":"run.terminal.completed","payload_schema_version":1,"payload":{"kind":"run_terminal","command_id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2","run_stream":{"kind":"run","id":"de066adf-c6de-4dbf-8afb-daa8b36b18c2"},"terminal":"completed","text":"pong","reason":null}} diff --git a/packages/server/test/fixtures/providers/muse/sessions/01a0ce3e-daea-76f0-99fc-d53076374955/session.jsonl b/packages/server/test/fixtures/providers/muse/sessions/01a0ce3e-daea-76f0-99fc-d53076374955/session.jsonl new file mode 100644 index 0000000..f5341bf --- /dev/null +++ b/packages/server/test/fixtures/providers/muse/sessions/01a0ce3e-daea-76f0-99fc-d53076374955/session.jsonl @@ -0,0 +1,9 @@ +{"schema_version":1,"id":"54aaad0d-e92f-420e-b94a-b16aef677c7f","stream":{"kind":"session","id":"01a0ce3e-daea-76f0-99fc-d53076374955"},"sequence":3,"recorded_at":1790166623248047,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session.metadata","payload_schema_version":1,"payload":{"kind":"metadata","record":{"workspace_root":"/tmp/muse-fx","provider_id":"meta","web_search_mode":"client","build":{"sha":"3c572bc734","semver":"1.3.0"},"tool_surface_version":"2"}}} +{"schema_version":1,"id":"948ea278-5fe1-4fcd-94a9-96ec2c7ea6ad","stream":{"kind":"session","id":"01a0ce3e-daea-76f0-99fc-d53076374955"},"sequence":4,"recorded_at":1790166623248295,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session.route_facts","payload_schema_version":2,"payload":{"kind":"route_facts","record":{"cwd":"/tmp/muse-fx","pid":77665,"terminal_kind":"Terminal","terminal_bundle_id":"com.apple.Terminal","terminal_title_identity":"d53076374955","local_runtime_command_socket_path":"/tmp/muse-share/runtime/muse/ms-e74e98bddfb2.sock"}}} +{"schema_version":1,"id":"1b5bdd68-3b2f-4023-9983-a2529ef3f93d","stream":{"kind":"session","id":"01a0ce3e-daea-76f0-99fc-d53076374955"},"sequence":5,"recorded_at":1790166623279853,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session.metadata","payload_schema_version":1,"payload":{"kind":"metadata","record":{"workspace_root":"/tmp/muse-fx","provider_id":"meta","model_id":"","web_search_mode":"client","build":{"sha":"3c572bc734","semver":"1.3.0"},"tool_surface_version":"2"}}} +{"schema_version":1,"id":"c865ef9f-31de-4677-8026-dce979dbfe72","stream":{"kind":"session","id":"01a0ce3e-daea-76f0-99fc-d53076374955"},"sequence":26,"recorded_at":1790166623411535,"record_type":"event","durability":"durable","causation_id":"7d810778-e657-45fb-a68f-3877cadc71ba","payload_type":"session.name.changed","payload_schema_version":1,"payload":{"authority_id":"ca1b6b99-7804-4649-ad99-1588f89a7241","operation_id":"b182f0d6-417f-495a-ac78-3c7e575b0244","session_id":"01a0ce3e-daea-76f0-99fc-d53076374955","previous_name":null,"new_name":"pastel-cepheus","source":"automatic"}} +{"schema_version":1,"id":"6033a766-8f78-4ad1-a2cf-ea416ddd0173","stream":{"kind":"session","id":"01a0ce3e-daea-76f0-99fc-d53076374955"},"sequence":35,"recorded_at":1790167312697074,"record_type":"event","durability":"durable","causation_id":"70a8a95a-b5fb-4951-8032-fdf4d9549deb","payload_type":"run.model.configured","payload_schema_version":1,"payload":{"kind":"run_model","record":{"command_id":"70a8a95a-b5fb-4951-8032-fdf4d9549deb","run_stream":{"kind":"run","id":"70a8a95a-b5fb-4951-8032-fdf4d9549deb"},"provider_id":"meta","profile_id":"tbh","model_id":"muse-spark-1.3-contributor","display_label":"muse-spark-1.3-contributor","source":"model_reconfigure"}}} +{"schema_version":1,"id":"daa3e46e-7099-4c1a-8cbc-a72891f9a1cf","stream":{"kind":"session","id":"01a0ce3e-daea-76f0-99fc-d53076374955"},"sequence":38,"recorded_at":1790167312697077,"record_type":"event","durability":"durable","causation_id":"70a8a95a-b5fb-4951-8032-fdf4d9549deb","payload_type":"runtime.user_intent.accepted","payload_schema_version":1,"payload":{"intent_id":"70a8a95a-b5fb-4951-8032-fdf4d9549deb","source_session_id":"01a0ce3e-daea-76f0-99fc-d53076374955","surface":"main","semantic_kind":{"kind":"chat"},"refill_blocks":[{"kind":"text","text":"hello world"}],"model_messages":[{"content":[{"kind":"text","text":"hello world"}]}],"bindings":[],"delivery_policy":"session_current","wake_policy":"start_once"}} +{"schema_version":1,"id":"4aca1f67-7355-47f2-8962-7ee231fe1883","stream":{"kind":"session","id":"01a0ce3e-daea-76f0-99fc-d53076374955"},"sequence":39,"recorded_at":1790167312728552,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"kind":"run","run_id":"70a8a95a-b5fb-4951-8032-fdf4d9549deb","event":{"kind":"started","prompt":"hello world"},"source_run_record_id":"72b8f3ce-5b99-4d50-8704-8f63703a1744","source_run_record_sequence":1}} +{"schema_version":1,"id":"9624bf65-12bb-431a-ac70-094f12147681","stream":{"kind":"session","id":"01a0ce3e-daea-76f0-99fc-d53076374955"},"sequence":72,"recorded_at":1790167317740164,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"kind":"run","run_id":"70a8a95a-b5fb-4951-8032-fdf4d9549deb","event":{"kind":"assistant_message_committed","message_id":"c9c408e7-0db8-41ce-b8e1-c19ce10c94df","response_id":"resp_6ab3c912d56b1669b0fa4f09","text":"Hello — I'm Muse Code powered by Meta Muse Spark. How can I help?","provider_item_id":"msg_01a0ce497bb075f68751589e9e3ed0b6"},"source_run_record_id":"76499caa-1772-47df-a3a2-f2465a58a89a","source_run_record_sequence":25}} +{"schema_version":1,"id":"c8f15c89-1109-4f36-bc2b-5674b6bcd7a6","stream":{"kind":"session","id":"01a0ce3e-daea-76f0-99fc-d53076374955"},"sequence":75,"recorded_at":1790167317764457,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"kind":"run","run_id":"70a8a95a-b5fb-4951-8032-fdf4d9549deb","event":{"kind":"model_completed","usage":{"input_tokens":23248,"output_tokens":141,"cached_tokens":5105,"cache_write_tokens":0,"cache_read_tokens":5105,"reasoning_tokens":115},"duration_ms":4876,"model":"muse-spark-1.3-contributor"},"source_run_record_id":"799ddc89-e032-4b34-9b43-0299e922c9db","source_run_record_sequence":27}} diff --git a/packages/server/test/fixtures/providers/muse/sessions/01a0ce3e-daea-76f0-99fc-d53076374955/subagent/39e5d5fc-1c2e-4acc-a294-741942f8fbbf/session.jsonl b/packages/server/test/fixtures/providers/muse/sessions/01a0ce3e-daea-76f0-99fc-d53076374955/subagent/39e5d5fc-1c2e-4acc-a294-741942f8fbbf/session.jsonl new file mode 100644 index 0000000..7244607 --- /dev/null +++ b/packages/server/test/fixtures/providers/muse/sessions/01a0ce3e-daea-76f0-99fc-d53076374955/subagent/39e5d5fc-1c2e-4acc-a294-741942f8fbbf/session.jsonl @@ -0,0 +1,3 @@ +{"schema_version":1,"id":"c0000001-0000-4000-8000-000000000001","stream":{"kind":"session","id":"39e5d5fc-1c2e-4acc-a294-741942f8fbbf"},"sequence":1,"recorded_at":1790167318000000,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session.metadata","payload_schema_version":1,"payload":{"kind":"metadata","record":{"workspace_root":"/tmp/muse-fx","display_name":"fixture-child"}}} +{"schema_version":1,"id":"c0000002-0000-4000-8000-000000000002","stream":{"kind":"session","id":"39e5d5fc-1c2e-4acc-a294-741942f8fbbf"},"sequence":2,"recorded_at":1790167318100000,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.user_intent.accepted","payload_schema_version":1,"payload":{"intent_id":"intent-child-1","refill_blocks":[{"text":"explore the muse provider adapters"}]}} +{"schema_version":1,"id":"c0000003-0000-4000-8000-000000000003","stream":{"kind":"session","id":"39e5d5fc-1c2e-4acc-a294-741942f8fbbf"},"sequence":3,"recorded_at":1790167318200000,"record_type":"event","durability":"durable","causation_id":null,"payload_type":"runtime.session","payload_schema_version":1,"payload":{"kind":"run","run_id":"39e5d5fc-1c2e-4acc-a294-741942f8fbbf","event":{"kind":"assistant_message_committed","message_id":"msg-child-1","text":"child reply from fixture"}}} diff --git a/packages/server/test/grok-provider.test.ts b/packages/server/test/grok-provider.test.ts index f29ce66..51fd535 100644 --- a/packages/server/test/grok-provider.test.ts +++ b/packages/server/test/grok-provider.test.ts @@ -165,4 +165,19 @@ describe("grok fixture files exist", () => { expect(text).not.toContain("/Users/"); } }); + + it("reads inject usage from usage.json", async () => { + const { readGrokUsageFile } = await import("../src/providers/grok/usage.js"); + const usagePath = path.join( + fixtureRoot, + "sessions", + "project", + noPlanSessionId, + "usage.json", + ); + expect(readGrokUsageFile(usagePath)).toEqual({ + inputTokens: 100, + outputTokens: 20, + }); + }); }); diff --git a/packages/server/test/harness-extras.test.ts b/packages/server/test/harness-extras.test.ts new file mode 100644 index 0000000..5c5a7e9 --- /dev/null +++ b/packages/server/test/harness-extras.test.ts @@ -0,0 +1,106 @@ +import { describe, expect, it } from "vitest"; +import fs from "node:fs"; +import path from "node:path"; +import { buildClaudeAgentArgv } from "../src/providers/claude-code/inject.js"; +import { buildAgyAgentArgv } from "../src/providers/antigravity/inject.js"; +import { execArgs as codexExecArgs } from "../src/providers/codex/inject.js"; +import { logHarnessExtrasLanes } from "../src/providers/harness-extras.js"; +import { ensureBareWorkspace } from "../src/providers/bare-workspace.js"; + +describe("harnessExtras argv", () => { + it("claude adds --disable-slash-commands when extras off", () => { + const args = buildClaudeAgentArgv({ + agent: "build", + prompt: "hi", + freshSessionId: "sess", + harnessExtras: false, + }); + expect(args).toContain("--disable-slash-commands"); + expect(args).not.toContain("--bare"); + }); + + it("claude omits skip flags when extras on", () => { + const args = buildClaudeAgentArgv({ + agent: "build", + prompt: "hi", + freshSessionId: "sess", + harnessExtras: true, + }); + expect(args).not.toContain("--bare"); + expect(args).not.toContain("--disable-slash-commands"); + }); +}); + +describe("ignoreLocalMarkdown argv", () => { + it("claude uses --setting-sources user (not --bare)", () => { + const args = buildClaudeAgentArgv({ + agent: "build", + prompt: "hi", + freshSessionId: "sess", + harnessExtras: false, + ignoreLocalMarkdown: true, + }); + expect(args).toContain("--setting-sources"); + expect(args[args.indexOf("--setting-sources") + 1]).toBe("user"); + expect(args).not.toContain("--bare"); + }); + + it("codex adds --skip-git-repo-check for bare cwd", () => { + const args = codexExecArgs({ + prompt: "hi", + ignoreLocalMarkdown: true, + }); + expect(args).toContain("--skip-git-repo-check"); + }); +}); + +describe("bare workspace", () => { + it("creates an empty pilot-bare dir", async () => { + const dir = await ensureBareWorkspace(); + expect(fs.existsSync(dir)).toBe(true); + expect(path.basename(dir)).toBe("pilot-bare"); + expect(fs.existsSync(path.join(dir, "AGENTS.md"))).toBe(false); + expect(fs.existsSync(path.join(dir, "CLAUDE.md"))).toBe(false); + }); +}); + +describe("logHarnessExtrasLanes", () => { + it("lists each muse lane as skipped when extras off", () => { + const lines: Array<[string, string]> = []; + logHarnessExtrasLanes((lane, line) => lines.push([lane, line]), "muse", false); + expect(lines[0]).toEqual(["meta", expect.stringMatching(/extras off · \d+ lanes? skipped/)]); + expect(lines.some(([, t]) => t.includes("⊘ skill-reminder · skipped"))).toBe(true); + expect(lines.some(([, t]) => t.includes("⊘ verify-reminder · skipped"))).toBe(true); + expect(lines.every(([, t]) => !t.includes(" · run") || t.startsWith("extras"))).toBe(true); + }); + + it("lists each muse lane as run when extras on", () => { + const lines: Array<[string, string]> = []; + logHarnessExtrasLanes((lane, line) => lines.push([lane, line]), "muse", true); + expect(lines[0]).toEqual(["meta", expect.stringMatching(/extras on · \d+ lanes?/)]); + expect(lines.some(([, t]) => t === "· skill-reminder · run")).toBe(true); + expect(lines.filter(([, t]) => t.includes(" · run")).length).toBeGreaterThanOrEqual(6); + }); + + it("lists claude slash-skills lane", () => { + const lines: Array<[string, string]> = []; + logHarnessExtrasLanes((lane, line) => lines.push([lane, line]), "claude-code", false); + expect(lines.some(([, t]) => t.includes("slash skills") && t.includes("skipped"))).toBe( + true, + ); + }); +}); + +describe("antigravity -p prompt attachment", () => { + it("passes the prompt as -p's value, not a trailing positional", () => { + const args = buildAgyAgentArgv({ + prompt: "hello", + harnessExtras: false, + }); + expect(args[0]).toBe("-p"); + expect(args[1]).toBe("hello"); + expect(args).toContain("--dangerously-skip-permissions"); + expect(args).toContain("--disable-slash-commands"); + expect(args.at(-1)).not.toBe("hello"); + }); +}); diff --git a/packages/server/test/inject-flags.test.ts b/packages/server/test/inject-flags.test.ts index 5280cf8..3649f01 100644 --- a/packages/server/test/inject-flags.test.ts +++ b/packages/server/test/inject-flags.test.ts @@ -6,7 +6,7 @@ describe("inject flag probes", () => { it("lists a probe for every provider inject surface", () => { const bins = new Set(INJECT_FLAG_PROBES.map((p) => p.bin)); expect(bins).toEqual( - new Set(["claude", "opencode", "cursor-agent", "agy", "codex", "copilot", "grok"]), + new Set(["claude", "opencode", "cursor-agent", "agy", "codex", "copilot", "grok", "muse"]), ); expect( INJECT_FLAG_PROBES.every((p) => p.match.length > 0 && p.why.length > 0), diff --git a/packages/server/test/job-session-ref.test.ts b/packages/server/test/job-session-ref.test.ts new file mode 100644 index 0000000..11b9bae --- /dev/null +++ b/packages/server/test/job-session-ref.test.ts @@ -0,0 +1,22 @@ +import { describe, expect, it } from "vitest"; +import { jobs } from "../src/jobs.js"; + +describe("JobRecord.sessionRef", () => { + it("stores sessionRef on create when provided", () => { + const { jobId } = jobs.create("run-session", "abc123", "g1", { + sessionRef: { provider: "claude-code", sessionId: "sess-full-id" }, + }); + const job = jobs.get(jobId); + expect(job?.sessionRef).toEqual({ + provider: "claude-code", + sessionId: "sess-full-id", + }); + jobs.finish(jobId, { status: "cancelled", error: "test cleanup" }); + }); + + it("omits sessionRef when not provided", () => { + const { jobId } = jobs.create("run-agent", "claude-code:x"); + expect(jobs.get(jobId)?.sessionRef).toBeUndefined(); + jobs.finish(jobId, { status: "cancelled", error: "test cleanup" }); + }); +}); diff --git a/packages/server/test/muse-provider.test.ts b/packages/server/test/muse-provider.test.ts new file mode 100644 index 0000000..aa1a91b --- /dev/null +++ b/packages/server/test/muse-provider.test.ts @@ -0,0 +1,270 @@ +import fs from "node:fs"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; +import { afterEach, describe, expect, it } from "vitest"; +import { + recordsFromLine, + microsToMs, + discoverSessions, + childrenOf, +} from "../src/providers/muse/discover.js"; +import { MuseProvider } from "../src/providers/muse/index.js"; +import { readMuseTranscript } from "../src/providers/muse/transcript.js"; + +const HERE = path.dirname(fileURLToPath(import.meta.url)); +const FIXTURE_SHARE = path.join(HERE, "fixtures/providers/muse"); +const SESSION_ID = "01a0ce3e-daea-76f0-99fc-d53076374955"; +const CHILD_ID = "39e5d5fc-1c2e-4acc-a294-741942f8fbbf"; +const FIXTURE = path.join(FIXTURE_SHARE, "sessions", SESSION_ID, "session.jsonl"); + +describe("muse provider parsers", () => { + it("microsToMs converts muse microsecond timestamps", () => { + expect(microsToMs(1790167312697077)).toBe(1790167312697); + expect(microsToMs(1_700_000_000_000)).toBe(1_700_000_000_000); + }); + + it("recordsFromLine yields outer + nested record_json children", () => { + const line = JSON.stringify({ + retained_frame: "session_permission_transaction", + children: [ + { + child_index: 0, + record_json: JSON.stringify({ + payload_type: "runtime.session.metadata", + payload: { record: { workspace_root: "/tmp/proj" } }, + }), + }, + ], + }); + const recs = [...recordsFromLine(line)]; + expect(recs.length).toBe(2); + expect(recs[1]?.payload_type).toBe("runtime.session.metadata"); + }); + + it("fixture session.jsonl contains user + assistant payload types", () => { + expect(fs.existsSync(FIXTURE)).toBe(true); + const raw = fs.readFileSync(FIXTURE, "utf8"); + expect(raw).toContain("runtime.user_intent.accepted"); + expect(raw).toContain("assistant_message_committed"); + expect(raw).toContain("muse-spark"); + }); +}); + +describe("muse provider discovery", () => { + const prev = process.env.MUSE_DATA_DIR; + + afterEach(() => { + if (prev === undefined) delete process.env.MUSE_DATA_DIR; + else process.env.MUSE_DATA_DIR = prev; + }); + + it("discovers fixture session and reads transcript", async () => { + process.env.MUSE_DATA_DIR = FIXTURE_SHARE; + const refs = await discoverSessions(); + expect(refs.length).toBe(1); + expect(refs[0]?.id).toBe(SESSION_ID); + expect(refs[0]?.provider).toBe("muse"); + expect(refs[0]?.model).toMatch(/muse-spark/); + + const msgs = await readMuseTranscript(SESSION_ID); + expect(msgs.length).toBeGreaterThanOrEqual(2); + expect(msgs.some((m) => m.role === "user")).toBe(true); + expect(msgs.some((m) => m.role === "assistant")).toBe(true); + }); +}); + +describe("muse session lineage", () => { + const prev = process.env.MUSE_DATA_DIR; + + afterEach(() => { + if (prev === undefined) delete process.env.MUSE_DATA_DIR; + else process.env.MUSE_DATA_DIR = prev; + }); + + it("lists subagent children under the parent and hides them from top-level", async () => { + process.env.MUSE_DATA_DIR = FIXTURE_SHARE; + const provider = new MuseProvider(); + const top = await provider.listSessions(); + expect(top.map((s) => s.id)).toEqual([SESSION_ID]); + expect(top.every((s) => !s.parentId)).toBe(true); + + const children = await provider.listChildren(SESSION_ID); + expect(children).toHaveLength(1); + expect(children[0]!.id).toBe(CHILD_ID); + expect(children[0]!.parentId).toBe(SESSION_ID); + expect(children[0]!.kind).toBe("subagent-run"); + expect(children[0]!.provider).toBe("muse"); + }); + + it("reads child transcript via parent→child lineage", async () => { + process.env.MUSE_DATA_DIR = FIXTURE_SHARE; + await discoverSessions(); + const kids = await childrenOf(SESSION_ID); + expect(kids[0]?.id).toBe(CHILD_ID); + + const msgs = await readMuseTranscript(CHILD_ID); + expect(msgs.length).toBeGreaterThanOrEqual(2); + expect(msgs.some((m) => m.parts.some((p) => p.text?.includes("explore the muse")))).toBe( + true, + ); + expect(msgs.some((m) => m.parts.some((p) => p.text?.includes("child reply")))).toBe(true); + }); + + it("sets tokenSource transcript and actualCost 0 when MSP usage is present", async () => { + process.env.MUSE_DATA_DIR = FIXTURE_SHARE; + const refs = await discoverSessions(); + const hit = refs.find((r) => r.id === SESSION_ID); + expect(hit?.actualCost).toBe(0); + expect(hit?.tokensIn).toBeGreaterThan(0); + expect(hit?.meta?.tokenSource).toBe("transcript"); + }); +}); + +describe("muse model soft catalog", () => { + it("parseMuseSparkIds extracts spark ids from help text", async () => { + const { parseMuseSparkIds } = await import("../src/providers/muse/inject.js"); + expect(parseMuseSparkIds("use --model muse-spark-1.3 or muse-spark-1.2")).toEqual([ + "muse-spark-1.3", + "muse-spark-1.2", + ]); + expect(parseMuseSparkIds("no models here")).toEqual([]); + }); + + it("listMuseModels honors MUSE_MODELS override", async () => { + const prev = process.env.MUSE_MODELS; + process.env.MUSE_MODELS = "muse-spark-lab, muse-spark-other"; + try { + const { listMuseModels } = await import("../src/providers/muse/inject.js"); + expect(await listMuseModels()).toEqual(["muse-spark-lab", "muse-spark-other"]); + } finally { + if (prev === undefined) delete process.env.MUSE_MODELS; + else process.env.MUSE_MODELS = prev; + } + }); +}); + +describe("muse jsonl ingest", () => { + it("reads session id from stream.id and terminal text", async () => { + const { ingestMuseJsonLine, createMuseIngestState } = await import( + "../src/providers/muse/inject.js" + ); + const state = createMuseIngestState(1_000); + const logs1 = ingestMuseJsonLine( + JSON.stringify({ + stream: { kind: "session", id: "01a0ce79-7535-7d92-bca9-b68e3bcf5a01" }, + payload_type: "run.output.delta", + payload: { text: "Hello" }, + }), + state, + ); + expect(state.sessionId).toBe("01a0ce79-7535-7d92-bca9-b68e3bcf5a01"); + expect(logs1.some(([lane]) => lane === "text")).toBe(true); + expect(logs1.some(([lane, t]) => lane === "meta" && t.includes("first answer"))).toBe(true); + const logs2 = ingestMuseJsonLine( + JSON.stringify({ + stream: { kind: "session", id: "01a0ce79-7535-7d92-bca9-b68e3bcf5a01" }, + payload_type: "run.terminal.completed", + payload: { text: "Hello, world." }, + }), + state, + ); + expect(state.resultText).toBe("Hello, world."); + expect(logs2.some(([lane, t]) => lane === "meta" && t.startsWith("■"))).toBe(true); + }); + + it("surfaces spawn/timing meta from exec --json fixture", async () => { + const { + ingestMuseJsonLine, + createMuseIngestState, + museExecEnv, + } = await import("../src/providers/muse/inject.js"); + const { formatDuration } = await import("../src/providers/run-metrics.js"); + expect(formatDuration(4500)).toBe("4.5s"); + + const off = museExecEnv(false); + expect(off.MUSE_EXPERIMENTAL_SKILL_REMINDER).toBe("0"); + expect(off.MUSE_EXPERIMENTAL_VERIFY_REMINDER).toBe("0"); + const on = museExecEnv(true); + expect(on.MUSE_EXPERIMENTAL_SKILL_REMINDER).toBeUndefined(); + + const fixture = path.join(HERE, "fixtures/providers/muse/exec/pong.jsonl"); + expect(fs.existsSync(fixture)).toBe(true); + const state = createMuseIngestState(); + const all: Array<[string, string]> = []; + for (const line of fs.readFileSync(fixture, "utf8").split("\n")) { + if (!line.trim()) continue; + all.push(...ingestMuseJsonLine(line, state)); + } + expect(state.deltas.join("")).toContain("pong"); + expect(state.resultText).toBe("pong"); + expect(state.modelId).toMatch(/muse-spark/); + expect(state.spawnNames).toEqual( + expect.arrayContaining(["skill-reminder", "verify-reminder"]), + ); + const metas = all.filter(([lane]) => lane === "meta").map(([, t]) => t); + expect(metas.some((t) => t.includes("spawn skill-reminder"))).toBe(true); + expect(metas.some((t) => t.includes("spawn verify-reminder"))).toBe(true); + expect(metas.some((t) => t.startsWith("■") && t.includes("spawns"))).toBe(true); + expect(metas.some((t) => t.includes("first answer"))).toBe(true); + // Reminder spawns stay meta-only; non-reminder tasks get a tool lane. + const tools = all.filter(([lane]) => lane === "tool").map(([, t]) => t); + expect(tools.some((t) => /reminder/i.test(t))).toBe(false); + }); + + it("emits tool lane for non-reminder task spawn", async () => { + const { ingestMuseJsonLine, createMuseIngestState } = await import( + "../src/providers/muse/inject.js" + ); + const state = createMuseIngestState(1_000); + const taskId = "task-shell-1"; + ingestMuseJsonLine( + JSON.stringify({ + payload_type: "task.lifecycle.proposed", + payload: { + task_id: taskId, + event: { kind: "proposed", task_id: taskId, task_kind: "tool.shell" }, + }, + }), + state, + ); + const logs = ingestMuseJsonLine( + JSON.stringify({ + payload_type: "task.lifecycle.started", + payload: { + task_id: taskId, + event: { kind: "started", task_id: taskId }, + }, + }), + state, + ); + expect(logs.some(([lane, t]) => lane === "meta" && t.includes("spawn"))).toBe(true); + expect(logs.some(([lane]) => lane === "tool")).toBe(true); + expect(logs.some(([lane, t]) => lane === "tool" && /reminder/i.test(t))).toBe(false); + }); + + it("accumulates tokens from runtime.session model_completed usage", async () => { + const { ingestMuseJsonLine, createMuseIngestState, museRunSummary } = await import( + "../src/providers/muse/inject.js" + ); + const state = createMuseIngestState(1_000); + ingestMuseJsonLine( + JSON.stringify({ + stream: { kind: "session", id: "sess-1" }, + payload_type: "runtime.session", + payload: { + kind: "run", + event: { + kind: "model_completed", + usage: { input_tokens: 100, output_tokens: 12 }, + model: "muse-spark-1.3", + }, + }, + }), + state, + ); + expect(state.tokensIn).toBe(100); + expect(state.tokensOut).toBe(12); + expect(state.modelId).toBe("muse-spark-1.3"); + expect(museRunSummary(state, 1_500)).toMatch(/100→12 tok/); + }); +}); diff --git a/packages/server/test/pilot-sessions.test.ts b/packages/server/test/pilot-sessions.test.ts new file mode 100644 index 0000000..6097990 --- /dev/null +++ b/packages/server/test/pilot-sessions.test.ts @@ -0,0 +1,47 @@ +import { describe, expect, it, beforeEach, afterEach } from "vitest"; +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; + +describe("pilot-sessions store", () => { + let tmp: string; + let prev: string | undefined; + + beforeEach(() => { + tmp = fs.mkdtempSync(path.join(os.tmpdir(), "threadle-pilot-")); + prev = process.env.THREADLE_CONFIG_DIR; + process.env.THREADLE_CONFIG_DIR = tmp; + }); + + afterEach(() => { + if (prev === undefined) delete process.env.THREADLE_CONFIG_DIR; + else process.env.THREADLE_CONFIG_DIR = prev; + fs.rmSync(tmp, { recursive: true, force: true }); + }); + + it("marks and lists pilot sessions idempotently", async () => { + const { markPilotSession, readPilotSessionKeys, listPilotSessions } = + await import("../src/pilot-sessions/store.js"); + + await markPilotSession({ + provider: "muse", + sessionId: "sess-1", + caseId: "muse:default", + }); + await markPilotSession({ + provider: "muse", + sessionId: "sess-1", + caseId: "muse:default", + }); + await markPilotSession({ provider: "cursor", sessionId: "sess-2" }); + + const keys = await readPilotSessionKeys(); + expect(keys.has("muse:sess-1")).toBe(true); + expect(keys.has("cursor:sess-2")).toBe(true); + expect(keys.size).toBe(2); + + const list = await listPilotSessions(); + expect(list).toHaveLength(2); + expect(list.find((e) => e.sessionId === "sess-1")?.caseId).toBe("muse:default"); + }); +}); diff --git a/packages/server/test/pilot.test.ts b/packages/server/test/pilot.test.ts new file mode 100644 index 0000000..84bd1d2 --- /dev/null +++ b/packages/server/test/pilot.test.ts @@ -0,0 +1,119 @@ +import { describe, expect, it, vi } from "vitest"; +import { allPilotCases, pilotCasesFor } from "../src/providers/pilot-defaults.js"; +import { PROVIDER_HARNESS_EXTRAS } from "@threadle/shared"; + +vi.mock("../src/providers/registry.js", () => ({ + registry: { + info: async () => [ + { id: "claude-code", available: true }, + { id: "opencode", available: false }, + { id: "cursor", available: true }, + { id: "antigravity", available: false }, + { id: "codex", available: false }, + { id: "copilot", available: false }, + { id: "grok", available: false }, + { id: "muse", available: true }, + ], + }, +})); + +describe("pilot case catalog", () => { + it("base mode is one case per provider; extras add flag variants", () => { + const base = pilotCasesFor(false); + expect(base.every((c) => !c.extra)).toBe(true); + expect(new Set(base.map((c) => c.provider)).size).toBe(8); + + const all = pilotCasesFor(true); + expect(all.length).toBeGreaterThan(base.length); + expect(all.some((c) => c.id === "codex:sandbox-ro")).toBe(true); + expect(all.some((c) => c.id === "claude-code:permission-plan")).toBe(true); + expect(allPilotCases().filter((c) => c.extra).length).toBeGreaterThan(0); + }); + + it("includes extras:off / extras:on pairs for harness providers", () => { + const all = pilotCasesFor(true); + const harnessProviders = Object.keys(PROVIDER_HARNESS_EXTRAS); + expect(harnessProviders.length).toBeGreaterThan(0); + for (const p of harnessProviders) { + const off = all.find((c) => c.id === `${p}:extras-off`); + const on = all.find((c) => c.id === `${p}:extras-on`); + expect(off, `${p}:extras-off`).toBeDefined(); + expect(on, `${p}:extras-on`).toBeDefined(); + expect(off?.harnessExtras).toBe(false); + expect(on?.harnessExtras).toBe(true); + expect(off?.extra).toBe(true); + expect(on?.extra).toBe(true); + expect(off?.variant).toBe("extras:off"); + expect(on?.variant).toBe("extras:on"); + } + // Base smoke stays extras-off; pairs are only under extra tests. + expect(pilotCasesFor(false).some((c) => c.variant.startsWith("extras:"))).toBe( + false, + ); + }); +}); + +describe("GET /api/run/pilot", () => { + it("returns base cases without ?extra", async () => { + const { runRoutes } = await import("../src/routes/run.js"); + const res = await runRoutes.request("/pilot"); + expect(res.status).toBe(200); + const body = (await res.json()) as { + prompt: string; + extraTests: boolean; + extraCaseCount: number; + cases: Array<{ + id: string; + provider: string; + variant: string; + available: boolean; + extra?: boolean; + model?: string; + skipReason?: string; + }>; + }; + expect(body.extraTests).toBe(false); + expect(body.extraCaseCount).toBeGreaterThan(0); + expect(body.prompt.toLowerCase()).toContain("ok"); + expect(body.cases).toHaveLength(8); + expect(body.cases.every((c) => !c.extra)).toBe(true); + + const byId = Object.fromEntries(body.cases.map((c) => [c.id, c])); + expect(byId["claude-code"]?.available).toBe(true); + expect(byId["claude-code"]?.model).toBe("haiku"); + expect(byId.cursor?.variant).toBe("default"); + expect(byId.opencode?.available).toBe(false); + expect(byId.opencode?.skipReason).toMatch(/not available/i); + }); + + it("includes flag variants with ?extra=1", async () => { + const { runRoutes } = await import("../src/routes/run.js"); + const res = await runRoutes.request("/pilot?extra=1"); + expect(res.status).toBe(200); + const body = (await res.json()) as { + extraTests: boolean; + cases: Array<{ + id: string; + sandbox?: string; + permissionMode?: string; + harnessExtras?: boolean; + extra?: boolean; + note?: string; + }>; + }; + expect(body.extraTests).toBe(true); + expect(body.cases.length).toBeGreaterThan(8); + const sand = body.cases.find((c) => c.id === "codex:sandbox-ro"); + expect(sand?.sandbox).toBe("read-only"); + expect(sand?.extra).toBe(true); + const perm = body.cases.find((c) => c.id === "claude-code:permission-plan"); + expect(perm?.permissionMode).toBe("plan"); + + const museOff = body.cases.find((c) => c.id === "muse:extras-off"); + const museOn = body.cases.find((c) => c.id === "muse:extras-on"); + expect(museOff?.harnessExtras).toBe(false); + expect(museOn?.harnessExtras).toBe(true); + expect(museOff?.note).toMatch(/skill-reminder/i); + expect(museOn?.note).toMatch(/skill-reminder/i); + }); +}); diff --git a/packages/server/test/provider-fixtures.test.ts b/packages/server/test/provider-fixtures.test.ts index 14b338c..bfc987d 100644 --- a/packages/server/test/provider-fixtures.test.ts +++ b/packages/server/test/provider-fixtures.test.ts @@ -16,6 +16,8 @@ import { _resetDbForTests } from "../src/providers/opencode/db.js"; import { readOpencodeTranscript } from "../src/providers/opencode/normalize.js"; import { discoverSessions } from "../src/providers/grok/discover.js"; import { readGrokTranscript } from "../src/providers/grok/transcript.js"; +import { discoverSessions as discoverMuseSessions, childrenOf as museChildrenOf } from "../src/providers/muse/discover.js"; +import { readMuseTranscript } from "../src/providers/muse/transcript.js"; const fixturesRoot = path.join( path.dirname(fileURLToPath(import.meta.url)), @@ -96,6 +98,27 @@ describe("provider golden fixtures", () => { delete process.env.OPENCODE_DATA_DIR; }); + it("accumulates step-finish tokens from JSON stream lines", async () => { + const { findOpencodeUsageInJsonl } = await import( + "../src/providers/opencode/inject.js" + ); + const stream = [ + JSON.stringify({ type: "text", part: { type: "text", text: "hi" } }), + JSON.stringify({ + type: "step-finish", + tokens: { input: 10, output: 5 }, + }), + JSON.stringify({ + type: "step-finish", + tokens: { input: 3, output: 1 }, + }), + ].join("\n"); + expect(findOpencodeUsageInJsonl(stream)).toEqual({ + inputTokens: 13, + outputTokens: 6, + }); + }); + it("reads known part types and skips unknown from fixture schema", async () => { tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), "threadle-oc-fix-")); process.env.OPENCODE_DATA_DIR = tmpDir; @@ -226,4 +249,41 @@ describe("provider golden fixtures", () => { ); }); }); + + describe("muse", () => { + const prev = process.env.MUSE_DATA_DIR; + const sessionId = "01a0ce3e-daea-76f0-99fc-d53076374955"; + + afterEach(() => { + if (prev === undefined) delete process.env.MUSE_DATA_DIR; + else process.env.MUSE_DATA_DIR = prev; + }); + + it("discovers session and skips unknown MSP payload types", async () => { + process.env.MUSE_DATA_DIR = fixture("muse"); + const refs = await discoverMuseSessions(); + expect(refs.length).toBe(1); + expect(refs[0]?.id).toBe(sessionId); + const msgs = await readMuseTranscript(sessionId); + expect(msgs.length).toBeGreaterThanOrEqual(2); + expect(msgs.every((m) => m.role === "user" || m.role === "assistant")).toBe(true); + expect( + msgs.every((m) => + m.parts.every((p) => p.type === "text" || p.type === "thinking" || p.type === "tool_use" || p.type === "tool_result"), + ), + ).toBe(true); + }); + + it("links subagent under parent via sessions//subagent/", async () => { + process.env.MUSE_DATA_DIR = fixture("muse"); + const childId = "39e5d5fc-1c2e-4acc-a294-741942f8fbbf"; + const refs = await discoverMuseSessions(); + expect(refs.map((r) => r.id)).toEqual([sessionId]); + const kids = await museChildrenOf(sessionId); + expect(kids).toHaveLength(1); + expect(kids[0]?.id).toBe(childId); + expect(kids[0]?.parentId).toBe(sessionId); + expect(kids[0]?.kind).toBe("subagent-run"); + }); + }); }); diff --git a/packages/server/test/run-phase.test.ts b/packages/server/test/run-phase.test.ts new file mode 100644 index 0000000..127cc78 --- /dev/null +++ b/packages/server/test/run-phase.test.ts @@ -0,0 +1,48 @@ +import { describe, expect, it } from "vitest"; +import { deriveRunPhase } from "@threadle/shared"; + +describe("deriveRunPhase", () => { + it("defaults to starting with no logs", () => { + expect(deriveRunPhase([])).toEqual({ phase: "starting", label: "starting" }); + }); + + it("maps thinking / tool / text lanes", () => { + expect( + deriveRunPhase([{ lane: "thinking", line: "hmm" }]).phase, + ).toBe("thinking"); + expect( + deriveRunPhase([{ lane: "tool", line: "Read: /tmp/a.ts" }]), + ).toEqual({ phase: "tool", label: "tool · Read" }); + expect( + deriveRunPhase([{ lane: "text", line: "ok" }]).phase, + ).toBe("answering"); + }); + + it("prefers the latest meaningful log", () => { + const logs = [ + { lane: "meta", line: "run started" }, + { lane: "thinking", line: "…" }, + { lane: "tool", line: "Bash: ls" }, + { lane: "text", line: "done" }, + ]; + expect(deriveRunPhase(logs)).toEqual({ + phase: "answering", + label: "answering", + }); + }); + + it("maps terminal and extra meta", () => { + expect( + deriveRunPhase([{ lane: "meta", line: "■ 1.2s · 10→2 tok" }]).phase, + ).toBe("done"); + expect( + deriveRunPhase([{ lane: "meta", line: "⎇ spawn skill-reminder" }]).phase, + ).toBe("extra"); + }); + + it("maps raw errors", () => { + expect( + deriveRunPhase([{ lane: "raw", line: "✗ boom" }]).phase, + ).toBe("error"); + }); +}); diff --git a/packages/server/test/settings.test.ts b/packages/server/test/settings.test.ts index 7817a97..ea7f6b8 100644 --- a/packages/server/test/settings.test.ts +++ b/packages/server/test/settings.test.ts @@ -187,3 +187,38 @@ describe("settings notifications", () => { expect(again.notifications?.handoff).toBe(false); }); }); + +describe("settings harnessExtras", () => { + it("defaults to false and round-trips true via PUT", async () => { + const { readSettings, settingsRoutes } = await import("../src/routes/settings.js"); + expect((await readSettings()).harnessExtras).toBe(false); + + const put = await settingsRoutes.request("/", { + method: "PUT", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ + editor: { mode: "vscode" }, + harnessExtras: true, + }), + }); + expect(put.status).toBe(200); + const body = (await put.json()) as { harnessExtras?: boolean }; + expect(body.harnessExtras).toBe(true); + expect((await readSettings()).harnessExtras).toBe(true); + }); + + it("accepts legacy museReminders alias", async () => { + const { readSettings, settingsRoutes } = await import("../src/routes/settings.js"); + const put = await settingsRoutes.request("/", { + method: "PUT", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ + editor: { mode: "vscode" }, + museReminders: true, + }), + }); + expect(put.status).toBe(200); + expect((await put.json() as { harnessExtras?: boolean }).harnessExtras).toBe(true); + expect((await readSettings()).harnessExtras).toBe(true); + }); +}); diff --git a/packages/server/test/skill-invoke.test.ts b/packages/server/test/skill-invoke.test.ts index aa41c53..e65ede3 100644 --- a/packages/server/test/skill-invoke.test.ts +++ b/packages/server/test/skill-invoke.test.ts @@ -6,6 +6,7 @@ const runOpencodeAgent = vi.fn(); const runCodexAgent = vi.fn(); const runCopilotAgent = vi.fn(); const runGrokAgent = vi.fn(); +const runMuseAgent = vi.fn(); const runAntigravityAgent = vi.fn(); vi.mock("../src/providers/claude-code/inject.js", () => ({ @@ -26,6 +27,9 @@ vi.mock("../src/providers/copilot/inject.js", () => ({ vi.mock("../src/providers/grok/inject.js", () => ({ runGrokAgent: (...a: unknown[]) => runGrokAgent(...a), })); +vi.mock("../src/providers/muse/inject.js", () => ({ + runMuseAgent: (...a: unknown[]) => runMuseAgent(...a), +})); vi.mock("../src/providers/antigravity/inject.js", () => ({ runAntigravityAgent: (...a: unknown[]) => runAntigravityAgent(...a), })); @@ -43,6 +47,7 @@ beforeEach(() => { runCodexAgent, runCopilotAgent, runGrokAgent, + runMuseAgent, runAntigravityAgent, ]) { fn.mockReset(); @@ -74,6 +79,7 @@ describe("runSkillInvoke providers", () => { ["codex", runCodexAgent], ["copilot", runCopilotAgent], ["grok", runGrokAgent], + ["muse", runMuseAgent], ["antigravity", runAntigravityAgent], ] as const)("dispatches %s", async (provider, fn) => { await runSkillInvoke({ ...base, provider, extra: "ctx" }); diff --git a/packages/server/test/token-baseline.test.ts b/packages/server/test/token-baseline.test.ts new file mode 100644 index 0000000..f38c8e4 --- /dev/null +++ b/packages/server/test/token-baseline.test.ts @@ -0,0 +1,24 @@ +import { describe, expect, it } from "vitest"; +import { + estimateTokenBaseline, + formatBaselineSuffix, + formatTokCompact, +} from "@threadle/shared"; + +describe("token baseline estimate", () => { + it("subtracts prompt (chars÷4) from tokensIn", () => { + const prompt = "Reply with exactly the single word: ok"; + const b = estimateTokenBaseline(6382, prompt, { ignoreLocalMarkdown: true }); + expect(b).toBeDefined(); + expect(b!.promptEst).toBe(Math.round(prompt.length / 4)); + expect(b!.baselineEst).toBe(6382 - b!.promptEst); + expect(b!.baselineNote).toMatch(/local md skipped/); + expect(formatBaselineSuffix(b!)).toMatch(/baseline ~/); + expect(formatTokCompact(6382)).toBe("6.4k"); + }); + + it("notes project md when not bare", () => { + const b = estimateTokenBaseline(8000, "ok", { ignoreLocalMarkdown: false }); + expect(b!.baselineNote).toMatch(/project md/); + }); +}); diff --git a/packages/shared/package.json b/packages/shared/package.json index c3eb9e7..b11bbc6 100644 --- a/packages/shared/package.json +++ b/packages/shared/package.json @@ -1,6 +1,6 @@ { "name": "@threadle/shared", - "version": "1.0.11", + "version": "1.1.1", "private": true, "type": "module", "main": "./src/index.ts", diff --git a/packages/shared/src/api.ts b/packages/shared/src/api.ts index 87f7076..30ad226 100644 --- a/packages/shared/src/api.ts +++ b/packages/shared/src/api.ts @@ -37,6 +37,11 @@ export interface InjectResult { newSessionId: string; provider: ProviderId; resultText?: string; + /** Optional first-turn usage (when the CLI reports it). */ + usage?: { + inputTokens?: number; + outputTokens?: number; + }; } export interface DistillJobAccepted { @@ -59,6 +64,19 @@ export interface RunAgentRequest { sandbox?: string; /** Codex ask-for-approval mode */ askForApproval?: string; + /** Override Settings harnessExtras for this spawn (provider-specific skips). */ + harnessExtras?: boolean; + /** @deprecated use harnessExtras */ + museReminders?: boolean; + /** + * Skip project AGENTS.md / CLAUDE.md / rules: run in an empty bare workspace + * (+ provider flags where available). Useful for cheap smoke / test-pilot. + */ + ignoreLocalMarkdown?: boolean; + /** Mark the resulting session as a Settings test-pilot smoke run. */ + pilot?: boolean; + /** Optional pilot case id recorded with the mark (e.g. muse:extras-off). */ + pilotCaseId?: string; } export interface RunSessionRequest { @@ -86,8 +104,8 @@ export type ServerEvent = | { type: "job.log"; jobId: string; - /** stream lane: assistant text, reasoning, tool call, or raw process line */ - lane: "text" | "thinking" | "tool" | "raw"; + /** stream lane: assistant text, reasoning, tool call, raw process, or run metrics */ + lane: "text" | "thinking" | "tool" | "raw" | "meta"; line: string; } | { type: "job.done"; jobId: string; payload?: ContextPayload; inject?: InjectResult } diff --git a/packages/shared/src/execution-manifest.ts b/packages/shared/src/execution-manifest.ts index da0133b..d450631 100644 --- a/packages/shared/src/execution-manifest.ts +++ b/packages/shared/src/execution-manifest.ts @@ -44,6 +44,13 @@ export function summarizeGraphExecution(g: Graph): ManifestItem[] { pushFlag(flags, "sandbox", d.sandbox); pushFlag(flags, "approval", d.askForApproval); pushFlag(flags, "model", d.model); + if (d.ref.provider === "muse" || d.ref.provider === "claude-code" || d.ref.provider === "grok" || d.ref.provider === "antigravity") { + const he = d.harnessExtras ?? d.museReminders; + if (he !== undefined) { + flags.push(he ? "extras: on" : "extras: off"); + } + } + if (d.ignoreLocalMarkdown) flags.push("ignore local md"); out.push({ ...base, label: d.label || d.ref.name, diff --git a/packages/shared/src/graph.ts b/packages/shared/src/graph.ts index dda5e4a..1f0a66e 100644 --- a/packages/shared/src/graph.ts +++ b/packages/shared/src/graph.ts @@ -71,6 +71,19 @@ export interface AgentDefNodeData { sandbox?: string; /** Codex: ask-for-approval mode (e.g. never | on-request | on-failure) */ askForApproval?: string; + /** + * Post-answer harness extras (Muse reminders, Claude hooks/slash skills, + * Grok subagents, Antigravity slash). When set, overrides Settings + * `harnessExtras` for this node. Omit to inherit. + */ + harnessExtras?: boolean; + /** @deprecated use harnessExtras */ + museReminders?: boolean; + /** + * Skip project AGENTS.md / CLAUDE.md / rules — run in an empty bare + * workspace (+ provider skip flags). Default off (use real project). + */ + ignoreLocalMarkdown?: boolean; } /** Value types flowing along text lanes */ @@ -940,6 +953,11 @@ const nodeDataSchema = z.union([ .enum(["never", "on-request", "on-failure", "untrusted"]) .or(z.literal("")) .optional(), + /** Post-answer extras; omit = inherit Settings. Legacy: museReminders. */ + harnessExtras: z.boolean().optional(), + museReminders: z.boolean().optional(), + /** Skip project AGENTS.md / CLAUDE.md / rules (bare workspace). */ + ignoreLocalMarkdown: z.boolean().optional(), }), z.object({ type: z.literal("prompt"), @@ -965,6 +983,7 @@ const nodeDataSchema = z.union([ "codex", "copilot", "grok", + "muse", ]) .optional(), model: z.string().optional(), diff --git a/packages/shared/src/harness-extras.ts b/packages/shared/src/harness-extras.ts new file mode 100644 index 0000000..03d6f29 --- /dev/null +++ b/packages/shared/src/harness-extras.ts @@ -0,0 +1,70 @@ +import type { ProviderId } from "./session.js"; + +/** + * Post-answer / background harness extras per provider — things that hold the + * CLI open after the main answer (Muse reminders, Claude hooks, Grok subagents, …). + * Only providers with a known skip switch are listed. + */ +export interface HarnessExtrasSpec { + /** Short rows for the inspector list */ + lanes: readonly string[]; + /** One-line tooltip / note */ + blurb: string; +} + +export const PROVIDER_HARNESS_EXTRAS: Partial> = { + muse: { + lanes: [ + "skill-reminder", + "goal-reminder", + "verify-reminder", + "memory-reminder", + "todo-reminder", + "scope-reminder", + ], + blurb: "Muse post-answer reminder subagents (skill / goal / verify / …).", + }, + "claude-code": { + lanes: ["slash skills"], + blurb: "Claude --disable-slash-commands (skills / slash commands). Avoids --bare, which drops login.", + }, + grok: { + lanes: ["subagents"], + blurb: "Grok --no-subagents (blocks spawned child agents).", + }, + antigravity: { + lanes: ["slash skills"], + blurb: "Antigravity --disable-slash-commands in print mode.", + }, +}; + +export function providerHasHarnessExtras(provider: ProviderId | string): boolean { + return Boolean(PROVIDER_HARNESS_EXTRAS[provider as ProviderId]); +} + +/** Effective on/off: node override wins, else Settings default. */ +export function resolveHarnessExtras( + nodeValue: boolean | undefined, + settingsDefault: boolean, +): boolean { + return nodeValue !== undefined ? nodeValue : settingsDefault; +} + +/** Read node field (supports legacy `museReminders`). */ +export function nodeHarnessExtrasField(data: { + harnessExtras?: boolean; + museReminders?: boolean; +}): boolean | undefined { + if (data.harnessExtras !== undefined) return data.harnessExtras; + if (data.museReminders !== undefined) return data.museReminders; + return undefined; +} + +export function setNodeHarnessExtras( + data: { harnessExtras?: boolean; museReminders?: boolean }, + value: boolean | undefined, +): void { + if (value === undefined) delete data.harnessExtras; + else data.harnessExtras = value; + if ("museReminders" in data) delete data.museReminders; +} diff --git a/packages/shared/src/index.ts b/packages/shared/src/index.ts index ccb27d1..6b88243 100644 --- a/packages/shared/src/index.ts +++ b/packages/shared/src/index.ts @@ -1,4 +1,5 @@ export * from "./session.js"; +export * from "./token-baseline.js"; export * from "./context.js"; export * from "./graph.js"; export * from "./execution-manifest.js"; @@ -39,6 +40,8 @@ export * from "./wait-idle.js"; export * from "./iterator.js"; export * from "./ply.js"; export * from "./api.js"; +export * from "./harness-extras.js"; +export * from "./run-phase.js"; export * from "./detached.js"; export * from "./frames.js"; export * from "./workflow.js"; diff --git a/packages/shared/src/run-phase.ts b/packages/shared/src/run-phase.ts new file mode 100644 index 0000000..de747a5 --- /dev/null +++ b/packages/shared/src/run-phase.ts @@ -0,0 +1,91 @@ +/** Coarse live phase for pilot / canvas while an agent job is running. */ +export type RunPhase = + | "starting" + | "thinking" + | "tool" + | "answering" + | "extra" + | "done" + | "error"; + +export interface RunPhaseLog { + lane: string; + line: string; +} + +export interface DerivedRunPhase { + phase: RunPhase; + /** Short UI label, e.g. `tool · Read` */ + label: string; +} + +const PHASE_LABEL: Record = { + starting: "starting", + thinking: "thinking", + tool: "tool", + answering: "answering", + extra: "extra", + done: "done", + error: "error", +}; + +function toolLabel(line: string): string { + // Prefer `name` from `name: detail` (toolSummary); else first token. + const colon = line.indexOf(":"); + const name = + colon > 0 + ? line.slice(0, colon).trim() + : line.trim().split(/\s/)[0]?.trim() || line.trim(); + const short = name.length > 28 ? `${name.slice(0, 28)}…` : name; + return short ? `tool · ${short}` : "tool"; +} + +function classifyLine(lane: string, line: string): RunPhase | undefined { + const text = line.trim(); + const lower = text.toLowerCase(); + + if (lane === "raw") { + if (/^✗|error|fail|traceback|exception/i.test(text)) return "error"; + return undefined; + } + + if (lane === "meta") { + if (/^■/.test(text) || /\b■\b/.test(text)) return "done"; + if (/^✗|error|fail/i.test(text)) return "error"; + if (/spawn |extras |harness extras|⊘ |reminders /i.test(text)) return "extra"; + if (/first answer|run started|lifecycle|model |opening |configured/i.test(text)) { + return "starting"; + } + return undefined; + } + + if (lane === "thinking") return "thinking"; + if (lane === "tool") return "tool"; + if (lane === "text") return "answering"; + + // Unknown lanes: treat as answering if it looks like content. + if (text && !lower.startsWith("▶")) return "answering"; + return undefined; +} + +/** + * Derive the current run phase from the latest meaningful job.log lines. + * Prefers the most recent classifiable lane; falls back to `starting`. + */ +export function deriveRunPhase(logs: readonly RunPhaseLog[]): DerivedRunPhase { + if (!logs.length) { + return { phase: "starting", label: PHASE_LABEL.starting }; + } + + for (let i = logs.length - 1; i >= 0; i--) { + const { lane, line } = logs[i]!; + const phase = classifyLine(lane, line); + if (!phase) continue; + if (phase === "tool") { + return { phase, label: toolLabel(line) }; + } + return { phase, label: PHASE_LABEL[phase] }; + } + + return { phase: "starting", label: PHASE_LABEL.starting }; +} diff --git a/packages/shared/src/session.ts b/packages/shared/src/session.ts index 02774d2..7c4b9ae 100644 --- a/packages/shared/src/session.ts +++ b/packages/shared/src/session.ts @@ -5,7 +5,8 @@ export type ProviderId = | "antigravity" | "codex" | "copilot" - | "grok"; + | "grok" + | "muse"; export type SessionStatus = "idle" | "live" | "running" | "waiting" | "unknown"; export type SessionKind = "session" | "subagent-run"; @@ -40,6 +41,13 @@ export interface SessionRef { meta?: Record; } +/** True when SessionRef was tagged by Settings → test pilot (or bare pilot cwd). */ +export function isPilotSession(s: { + meta?: Record; +}): boolean { + return s.meta?.pilot === true; +} + export type ContentPartType = | "text" | "thinking" diff --git a/packages/shared/src/token-baseline.ts b/packages/shared/src/token-baseline.ts new file mode 100644 index 0000000..6e001da --- /dev/null +++ b/packages/shared/src/token-baseline.ts @@ -0,0 +1,46 @@ +/** + * Providers rarely expose a true system/tools/rules token split. For smoke + * runs we approximate: baseline ≈ tokensIn − prompt (chars÷4). + */ + +export interface TokenBaselineEstimate { + tokensIn: number; + promptEst: number; + baselineEst: number; + /** What’s left after subtracting the user prompt. */ + baselineNote: string; +} + +/** Same rule of thumb Claude / many CLIs use (UTF-8 chars ÷ 4). */ +export function estimateCharsToTokens(chars: number): number { + if (!Number.isFinite(chars) || chars <= 0) return 0; + return Math.max(1, Math.round(chars / 4)); +} + +export function estimateTokenBaseline( + tokensIn: number, + promptText: string, + opts?: { ignoreLocalMarkdown?: boolean }, +): TokenBaselineEstimate | undefined { + if (!Number.isFinite(tokensIn) || tokensIn <= 0) return undefined; + const promptEst = estimateCharsToTokens(promptText.trim().length); + const baselineEst = Math.max(0, Math.round(tokensIn) - promptEst); + const baselineNote = opts?.ignoreLocalMarkdown + ? "system · tools · MCP (local md skipped)" + : "system · tools · rules · skills · MCP · project md"; + return { tokensIn: Math.round(tokensIn), promptEst, baselineEst, baselineNote }; +} + +/** Compact “~6.4k” for meta lines. */ +export function formatTokCompact(n: number): string { + if (!Number.isFinite(n) || n < 0) return "?"; + if (n < 1000) return String(Math.round(n)); + const k = n / 1000; + return `${k >= 10 ? k.toFixed(0) : k.toFixed(1)}k`; +} + +/** Suffix for run-log `■ … tok` lines. */ +export function formatBaselineSuffix(b: TokenBaselineEstimate): string { + if (b.baselineEst <= 0) return ""; + return `baseline ~${formatTokCompact(b.baselineEst)} (${b.baselineNote})`; +} diff --git a/packages/web/package.json b/packages/web/package.json index 1ffc47e..dfb7aad 100644 --- a/packages/web/package.json +++ b/packages/web/package.json @@ -1,6 +1,6 @@ { "name": "@threadle/web", - "version": "1.0.11", + "version": "1.1.1", "private": true, "type": "module", "scripts": { diff --git a/packages/web/src/api/client.ts b/packages/web/src/api/client.ts index 42b8e7c..ee7f925 100644 --- a/packages/web/src/api/client.ts +++ b/packages/web/src/api/client.ts @@ -18,6 +18,7 @@ import type { WorkflowFolderIndex, FavoriteCreate, FavoriteEntry, + ProviderId, } from "@threadle/shared"; async function http(path: string, init?: RequestInit): Promise { @@ -238,19 +239,52 @@ export const api = { kind: string; label?: string; graphId?: string; + sessionRef?: { provider: string; sessionId: string }; status: "running" | "done" | "error" | "cancelled"; createdAt: number; finishedAt?: number; error?: string; + result?: { + inject?: { + provider: string; + newSessionId: string; + }; + }; }> >("/api/jobs"), cancelJob: (id: string) => http<{ ok: boolean }>(`/api/jobs/${id}`, { method: "DELETE" }), + jobLogs: (id: string) => + http<{ lines: Array<{ ts: number; lane: string; line: string }> }>( + `/api/jobs/${encodeURIComponent(id)}/logs`, + ), runAgent: (req: RunAgentRequest) => http<{ jobId: string }>("/api/run/agent", { method: "POST", body: JSON.stringify(req), }), + pilotPlan: (extraTests = false) => + http<{ + prompt: string; + extraTests: boolean; + harnessExtras: false; + extraCaseCount: number; + cases: Array<{ + id: string; + provider: ProviderId; + variant: string; + agent: string; + model?: string; + permissionMode?: string; + sandbox?: string; + askForApproval?: string; + harnessExtras?: boolean; + extra?: boolean; + note?: string; + available: boolean; + skipReason?: string; + }>; + }>(`/api/run/pilot${extraTests ? "?extra=1" : ""}`), runSession: (req: RunSessionRequest) => http<{ jobId: string }>("/api/run/session", { method: "POST", diff --git a/packages/web/src/canvas/nodes/AgentDefNode.vue b/packages/web/src/canvas/nodes/AgentDefNode.vue index cb5f35c..d33c704 100644 --- a/packages/web/src/canvas/nodes/AgentDefNode.vue +++ b/packages/web/src/canvas/nodes/AgentDefNode.vue @@ -1,5 +1,19 @@ - + - + on + + +

+ When on, the agent runs in an empty workspace so project markdown and + rules do not inflate input tokens. +

+
@@ -135,9 +174,16 @@ diff --git a/packages/web/src/panels/StatsCharts.vue b/packages/web/src/panels/StatsCharts.vue index ce89f99..df48b1f 100644 --- a/packages/web/src/panels/StatsCharts.vue +++ b/packages/web/src/panels/StatsCharts.vue @@ -128,7 +128,7 @@ const props = defineProps<{ sessions: SessionRef[] }>(); type Counts = Record; function emptyCounts(): Counts { - return { "claude-code": 0, opencode: 0, cursor: 0, antigravity: 0, codex: 0, copilot: 0, grok: 0 }; + return Object.fromEntries(PROVIDER_IDS.map((id) => [id, 0])) as Counts; } function pct(v: number, max: number): string { diff --git a/packages/web/src/panels/useNavItems.ts b/packages/web/src/panels/useNavItems.ts index 3242e44..79b489a 100644 --- a/packages/web/src/panels/useNavItems.ts +++ b/packages/web/src/panels/useNavItems.ts @@ -1,6 +1,7 @@ import { computed, onMounted, ref, type ComputedRef, type Ref } from "vue"; import { api } from "@/api/client"; import { useSessionsStore } from "@/stores/sessions"; +import { isPilotSession } from "@threadle/shared"; import { DEFAULT_NAV_ITEMS, type NavItem } from "./nav-items"; type Count = number | ""; @@ -179,7 +180,8 @@ export function useNavItems(opts?: { const baseCounts: Record string | number> = { workflows: () => workflowCount.value, runs: () => runsRunning.value, - sessions: () => sessions.sessions.length || blank(), + sessions: () => + sessions.sessions.filter((s) => !isPilotSession(s)).length || blank(), map: () => projectCount.value, lineage: () => lineageCount.value, activity: () => activityCount.value, diff --git a/packages/web/src/stores/settings.ts b/packages/web/src/stores/settings.ts index 81cf97c..b8e82c5 100644 --- a/packages/web/src/stores/settings.ts +++ b/packages/web/src/stores/settings.ts @@ -71,6 +71,8 @@ export const useSettingsStore = defineStore("settings", () => { const mcpClientEnabled = ref(true); const mcpDisabledServers = ref([]); const mcpPublishAllowlist = ref([]); + /** Post-answer harness extras; default off for workflow wall time. */ + const harnessExtras = ref(false); const notifications = reactive({ ...DEFAULT_NOTIFY }); const appearance = ref(readStoredAppearance()); /** Only non-default overrides are stored. */ @@ -114,6 +116,8 @@ export const useSettingsStore = defineStore("settings", () => { mcpClientEnabled?: boolean; mcpDisabledServers?: string[]; mcpPublishAllowlist?: string[] | null; + harnessExtras?: boolean; + museReminders?: boolean; }; if (s.editor?.mode) mode.value = s.editor.mode; command.value = s.editor?.command ?? ""; @@ -127,6 +131,7 @@ export const useSettingsStore = defineStore("settings", () => { : Array.isArray(s.mcpPublishAllowlist) ? s.mcpPublishAllowlist : []; + harnessExtras.value = s.harnessExtras === true || s.museReminders === true; Object.assign(notifications, parseNotify(s)); if (isAppearance(s.appearance)) { appearance.value = s.appearance; @@ -155,6 +160,7 @@ export const useSettingsStore = defineStore("settings", () => { mcpClientEnabled: mcpClientEnabled.value, mcpDisabledServers: mcpDisabledServers.value, mcpPublishAllowlist: mcpPublishAllowlist.value, + harnessExtras: harnessExtras.value, notifications: { ...notifications }, appearance: appearance.value, providerColors: Object.keys(providerColors.value).length @@ -192,6 +198,12 @@ export const useSettingsStore = defineStore("settings", () => { await save(); } + async function setHarnessExtras(next: boolean): Promise { + if (harnessExtras.value === next) return; + harnessExtras.value = next; + await save(); + } + const editorLabel = computed(() => mode.value === "command" ? (command.value.split(/\s+/)[0]?.split("/").pop() ?? "open") @@ -218,6 +230,8 @@ export const useSettingsStore = defineStore("settings", () => { mcpClientEnabled, mcpDisabledServers, mcpPublishAllowlist, + harnessExtras, + setHarnessExtras, notifications, notifyAny, notifyAllows, diff --git a/packages/web/src/theme/theme.css b/packages/web/src/theme/theme.css index 7096901..6ef4495 100644 --- a/packages/web/src/theme/theme.css +++ b/packages/web/src/theme/theme.css @@ -112,6 +112,7 @@ html[data-theme="dark"] { --codex: #5faf82; --copilot: #d489a8; --grok: #9b87c9; + --muse: #568ab8; --threadle: #8a91a8; --context: #cfa93c; --lane-session: #5f9fe8; @@ -513,6 +514,9 @@ body { .vue-flow__edge.prov-grok .vue-flow__edge-path { stroke: var(--grok); } +.vue-flow__edge.prov-muse .vue-flow__edge-path { + stroke: var(--muse); +} /* agent out:err — after provider color so the fuse arm stays red */ .vue-flow__edge.edge-err .vue-flow__edge-path { stroke: var(--status-error); @@ -670,6 +674,7 @@ body { .filter-chip[data-provider="codex"] { --chip-prov: var(--codex); } .filter-chip[data-provider="copilot"] { --chip-prov: var(--copilot); } .filter-chip[data-provider="grok"] { --chip-prov: var(--grok); } +.filter-chip[data-provider="muse"] { --chip-prov: var(--muse); } .filter-chip[data-provider="threadle"] { --chip-prov: var(--threadle); } .filter-chip:disabled { opacity: 0.4; diff --git a/packages/web/src/views/GraphEditor.vue b/packages/web/src/views/GraphEditor.vue index ed78093..c94afb0 100644 --- a/packages/web/src/views/GraphEditor.vue +++ b/packages/web/src/views/GraphEditor.vue @@ -207,6 +207,15 @@ :server-project-dir="serverProjectDir" @dismiss="dismissRecipeSetup" /> + jid === event.jobId); + if (jobEntry) jobProgress[jobEntry[0]] = phase; return; } // Detached mid-run paint: active node border + status without waiting for job.done. @@ -4475,7 +4494,8 @@ function onServerEvent(event: ServerEvent): void { const [nodeId] = entry; const node = store.nodeById(nodeId); if (event.type === "job.progress") { - jobProgress[nodeId] = event.message; + // Seed phase only — never clobber deriveRunPhase labels with the verbose message. + if (jobProgress[nodeId] === undefined) jobProgress[nodeId] = "starting"; } else if (event.type === "job.done") { pendingJobs.delete(nodeId); delete jobProgress[nodeId]; @@ -4955,6 +4975,13 @@ function setJobLabel(jobId: string, label: string): void { if (!oldest.done) jobLabels.delete(oldest.value); } } + +function logsForJob(jobId: string): Array<{ lane: string; line: string }> { + return logEntries.value + .filter((e) => e.jobId === jobId) + .map((e) => ({ lane: e.lane, line: e.line })); +} + const LOG_CAP = 1000; /** the server-side job registered for the current client-driven workflow run */ @@ -5074,6 +5101,8 @@ async function runAgent(): Promise { permissionMode: data.permissionMode, sandbox: data.sandbox, askForApproval: data.askForApproval, + harnessExtras: data.harnessExtras ?? data.museReminders, + ignoreLocalMarkdown: data.ignoreLocalMarkdown, projectDir: linkedLive?.projectDir ?? linkedData?.snapshot?.projectDir ?? @@ -5095,7 +5124,8 @@ function onRunEvent(event: ServerEvent): boolean { const agentNode = store.nodeById(agentNodeId); if (event.type === "job.progress") { - runProgress[agentNodeId] = event.message; + // Seed phase only — never clobber deriveRunPhase labels with the verbose message. + if (runProgress[agentNodeId] === undefined) runProgress[agentNodeId] = "starting"; return true; } pendingRuns.delete(event.jobId); @@ -5260,6 +5290,73 @@ const detachedJobs = new Map(); const detachedBusy = ref(false); let detachedPollTimer: ReturnType | undefined; +/** Top-right canvas HUD while any run path is live. */ +const canvasRunLive = computed( + () => graphRunning.value || detachedBusy.value || runBusy.value, +); +const canvasRunStartedAt = ref(); +watch( + canvasRunLive, + (live) => { + if (live) { + if (canvasRunStartedAt.value === undefined) canvasRunStartedAt.value = Date.now(); + } else { + canvasRunStartedAt.value = undefined; + } + }, + { immediate: true }, +); +const canvasRunTitle = computed(() => { + const running = activeRunningLabels.value; + if (running.length) return running.length === 1 ? running[0]! : `${running.length} nodes`; + if (runBusy.value) return "agent"; + if (detachedBusy.value) return "≫ detached"; + if (graphRunning.value) return "workflow"; + return "running"; +}); +const canvasRunDetail = computed(() => { + // Prefer phase from the most recent agent/workflow job logs. + const latestJobId = (() => { + for (let i = logEntries.value.length - 1; i >= 0; i--) { + const id = logEntries.value[i]!.jobId; + if (id && id !== "run") return id; + } + return undefined; + })(); + if (latestJobId && (runBusy.value || graphRunning.value || detachedBusy.value)) { + const { phase, label } = deriveRunPhase(logsForJob(latestJobId)); + if (phase !== "starting" || logsForJob(latestJobId).length > 0) { + return label; + } + } + for (let i = logEntries.value.length - 1; i >= 0; i--) { + const e = logEntries.value[i]!; + if (e.lane !== "meta") continue; + const line = e.line.replace(/^[^:]+:\s*/, ""); + if ( + /first answer|spawn |extras |reminders |✓ |model |opening |completed meta/i.test(line) + ) { + return line.length > 72 ? `${line.slice(0, 72)}…` : line; + } + } + const queued = activeQueuedLabels.value; + if (queued.length) return `queued · ${queued.slice(0, 2).join(", ")}`; + return undefined; +}); + +async function onHarnessExtrasToggle(next: boolean): Promise { + await settings.setHarnessExtras(next); + if (canvasRunLive.value) { + pushLog( + "run", + "meta", + next + ? "harness extras on — next agent spawn" + : "harness extras off — next agent spawn", + ); + } +} + const { tabRuns, tabRunPhase, @@ -6340,6 +6437,14 @@ function setCtxModel(m: string | undefined): void { ctxMenu.value = undefined; } +function openNodeDetails(): void { + const id = ctxMenu.value?.nodeId; + ctxMenu.value = undefined; + modelOpen.value = false; + exchangeOpen.value = false; + if (id) inspected.value = id; +} + function nodeLabelOf(n: GraphNode): string { switch (n.data.type) { case "prompt": @@ -7856,6 +7961,8 @@ async function runGraph(scope?: Set): Promise { permissionMode: agentData.permissionMode, sandbox: agentData.sandbox, askForApproval: agentData.askForApproval, + harnessExtras: agentData.harnessExtras ?? agentData.museReminders, + ignoreLocalMarkdown: agentData.ignoreLocalMarkdown, }); setJobLabel(jobId, agentData.ref.name); node.lastRunId = jobId; @@ -8004,7 +8111,7 @@ async function runGraph(scope?: Set): Promise { pushLog("run", "raw", `⋈ merge synthesize via ${node.data.provider}`); const { jobId } = await api.runAgent({ provider: node.data.provider, - agent: node.data.agent || (node.data.provider === "cursor" || node.data.provider === "antigravity" ? "agent" : node.data.provider === "codex" ? "codex" : node.data.provider === "copilot" ? "copilot" : node.data.provider === "grok" ? "grok" : "build"), + agent: node.data.agent || (node.data.provider === "cursor" || node.data.provider === "antigravity" ? "agent" : node.data.provider === "codex" ? "codex" : node.data.provider === "copilot" ? "copilot" : node.data.provider === "grok" ? "grok" : node.data.provider === "muse" ? "muse" : "build"), model: node.data.model, prompt: text, graphId: g.id, diff --git a/packages/web/src/views/dashboard/ActivityView.vue b/packages/web/src/views/dashboard/ActivityView.vue index 4c60ceb..decb224 100644 --- a/packages/web/src/views/dashboard/ActivityView.vue +++ b/packages/web/src/views/dashboard/ActivityView.vue @@ -487,7 +487,7 @@ import { useFileViewersStore, isLikelyTextPath } from "@/stores/fileViewers"; import "./chrome.css"; interface SessionActivity { - provider: "claude-code" | "opencode" | "cursor" | "antigravity" | "codex" | "copilot" | "grok"; + provider: ProviderId; sessionId: string; title?: string; projectDir: string; diff --git a/packages/web/src/views/dashboard/AgentsView.vue b/packages/web/src/views/dashboard/AgentsView.vue index bfde2b9..bc5d079 100644 --- a/packages/web/src/views/dashboard/AgentsView.vue +++ b/packages/web/src/views/dashboard/AgentsView.vue @@ -18,6 +18,8 @@ :placeholder=" agentBrowse === 'defs' ? 'Filter agents…' + : agentBrowse === 'skills' + ? 'Filter skills…' : agentBrowse === 'memory' ? 'Filter memory…' : agentBrowse === 'plugins' @@ -85,6 +87,14 @@ > plugins{{ pluginsCount != null ? ` · ${pluginsCount}` : "" }} +
@@ -132,7 +142,14 @@
{{ pickedRun.title ?? shortId(pickedRun.id) }}
- + sub ⟨/⟩ {{ pickedRun.agent }}
@@ -287,7 +304,14 @@
{{ pickedRun.title ?? shortId(pickedRun.id) }}
- + sub ⟨/⟩ {{ pickedRun.agent }}
@@ -337,6 +361,17 @@
+

+ {{ + agentBrowse === "skills" + ? agentFilter.trim() + ? "no skills match these filters" + : "no skill defs found — providers that expose SKILL.md as agents will appear here" + : agentFilter.trim() + ? "no agents match these filters" + : "no agent defs found" + }} +

@@ -455,7 +490,14 @@
{{ pickedRun.title ?? shortId(pickedRun.id) }}
- + sub ⟨/⟩ {{ pickedRun.agent }}
@@ -703,12 +745,13 @@ import PluginsBrowser from "@/panels/PluginsBrowser.vue"; import AgentsGraph, { type GraphPick } from "@/panels/AgentsGraph.vue"; import DetailExpandControls from "@/panels/DetailExpandControls.vue"; import DetailExpandModal from "@/panels/DetailExpandModal.vue"; +import { useJobPhases } from "@/lib/useJobPhases"; import "./chrome.css"; const props = defineProps<{ /** Deep-link: agent name (+ optional provider) from `?agent=` / `?provider=`. */ focus?: { name: string; provider?: string }; - /** Deep-link: `?browse=memory` / `?browse=plugins` etc. */ + /** Deep-link: `?browse=memory` / `?browse=plugins` / `?browse=skills` etc. */ browse?: string; /** Deep-link: `?plugin=provider:id` when browsing plugins. */ pluginFocus?: string; @@ -725,6 +768,7 @@ const settings = useSettingsStore(); const fileViewers = useFileViewersStore(); const favorites = useFavoritesStore(); void favorites.ensureLoaded(); +const { phaseForSession } = useJobPhases(); const runCtx = ref<{ x: number; y: number; session: SessionRef }>(); const defCtx = ref<{ x: number; y: number; agent: AgentDef }>(); @@ -832,13 +876,25 @@ const agentInstances = computed(() => ({ })); /** Exclusive Agents browse modes — tiles are radios, not toggles. */ -type AgentBrowse = "defs" | "runs" | "subs" | "live" | "graph" | "memory" | "plugins"; +type AgentBrowse = + | "defs" + | "runs" + | "subs" + | "live" + | "graph" + | "memory" + | "plugins" + | "skills"; const agentBrowse = ref("defs"); const pickedRun = ref(); const runDetailExpanded = ref(false); const memoryCount = ref(null); const pluginsCount = ref(null); +const skillDefCount = computed( + () => sessions.agents.filter((a) => a.kind === "skill").length, +); + function closePickedRun(): void { pickedRun.value = undefined; runDetailExpanded.value = false; @@ -852,6 +908,7 @@ const BROWSE_MODES: AgentBrowse[] = [ "graph", "memory", "plugins", + "skills", ]; function browseFromProp(raw?: string): AgentBrowse | undefined { @@ -879,7 +936,7 @@ function isSubRun(s: SessionRef): boolean { function setAgentBrowse(mode: AgentBrowse): void { agentBrowse.value = mode; - if (mode === "defs") { + if (mode === "defs" || mode === "skills") { pickedRun.value = undefined; } else if (mode === "memory" || mode === "plugins") { pickedRun.value = undefined; @@ -1195,7 +1252,9 @@ watch(agentProviders, (chips) => { const agentGroups = computed(() => { const providers = new Map>(); const q = agentFilter.value.toLowerCase(); + const skillsOnly = agentBrowse.value === "skills"; for (const a of sessions.agents) { + if (skillsOnly ? a.kind !== "skill" : a.kind === "skill") continue; if (agentProviderF.value !== "all" && a.provider !== agentProviderF.value) continue; if (q && !a.name.toLowerCase().includes(q) && !a.description?.toLowerCase().includes(q)) { continue; diff --git a/packages/web/src/views/dashboard/FavoritesView.vue b/packages/web/src/views/dashboard/FavoritesView.vue index 0fd3b6a..1862264 100644 --- a/packages/web/src/views/dashboard/FavoritesView.vue +++ b/packages/web/src/views/dashboard/FavoritesView.vue @@ -185,6 +185,7 @@ function inferProviderFromPath(raw: string): string | undefined { if (p.includes("/.codex/") || p.includes("/codex/")) return "codex"; if (p.includes("/copilot")) return "copilot"; if (p.includes("/.grok/") || p.includes("/grok/")) return "grok"; + if (p.includes("/.muse/") || p.includes("/.local/share/muse/")) return "muse"; return undefined; } diff --git a/packages/web/src/views/dashboard/MetaView.vue b/packages/web/src/views/dashboard/MetaView.vue index f913f71..0fa3a8e 100644 --- a/packages/web/src/views/dashboard/MetaView.vue +++ b/packages/web/src/views/dashboard/MetaView.vue @@ -764,6 +764,18 @@ const STORAGE_PATHS = [ access: "read-only", owner: "grok", }, + { + path: "~/.local/share/muse/sessions", + desc: "Muse Code sessions.", + access: "read-only", + owner: "muse", + }, + { + path: "~/.muse", + desc: "Muse Code config.", + access: "read-only", + owner: "muse", + }, ] as const; type StoragePath = (typeof STORAGE_PATHS)[number]; diff --git a/packages/web/src/views/dashboard/RunsView.vue b/packages/web/src/views/dashboard/RunsView.vue index ddd051a..a396372 100644 --- a/packages/web/src/views/dashboard/RunsView.vue +++ b/packages/web/src/views/dashboard/RunsView.vue @@ -72,7 +72,7 @@ {{ RUN_KIND_LABELS[j.kind] ?? j.kind }} - {{ j.status }} + {{ runStatusLabel(j) }} {{ relativeTime(j.createdAt) }} {{ runDuration(j) }} @@ -219,9 +219,10 @@ + + diff --git a/packages/web/src/views/graph-editor/GraphRunHud.vue b/packages/web/src/views/graph-editor/GraphRunHud.vue new file mode 100644 index 0000000..c1b7723 --- /dev/null +++ b/packages/web/src/views/graph-editor/GraphRunHud.vue @@ -0,0 +1,180 @@ + + + + + diff --git a/packages/web/src/views/graph-editor/NodeContextMenu.vue b/packages/web/src/views/graph-editor/NodeContextMenu.vue index 23c4a3a..bc55f3e 100644 --- a/packages/web/src/views/graph-editor/NodeContextMenu.vue +++ b/packages/web/src/views/graph-editor/NodeContextMenu.vue @@ -6,6 +6,13 @@ @contextmenu.prevent >
{{ label }}
+