Skip to content

fix(runner): tell the model its rendered-skills path so it stops guessing - #5904

Merged
mmabrouk merged 1 commit into
release/v0.112.0from
fix/skills-read-path-guidance
Aug 10, 2026
Merged

fix(runner): tell the model its rendered-skills path so it stops guessing#5904
mmabrouk merged 1 commit into
release/v0.112.0from
fix/skills-read-path-guidance

Conversation

@mmabrouk

@mmabrouk mmabrouk commented Aug 10, 2026

Copy link
Copy Markdown
Member

Context

In a live session (debugged today), an agent asked about its own skills emitted a fully-formed absolute skills path from pattern memory as its very first tool call. The guess was structurally wrong (one id segment where the real mount path has two), so the read failed with ENOENT; the model then self-corrected by listing its actual working directory, which cost a failed call plus an approval interruption before it found the skill exactly where it had been correctly materialized all along.

The gap: the platform guidance names the durable agent-files/ folder with its real absolute path, but nothing states where the rendered skills live, so the model is left to infer that path and sometimes guesses.

Changes

A new skills-read-path appendix in the platform guidance states the harness-correct absolute skills directory and says to list it rather than construct paths from memory. The path mirrors prepareWorkspace's materialization split: Pi reads the immutable agents/skills/<digest> snapshot; other harnesses read .{acpAgent}/skills/<name>. The appendix only appears when the run actually materialized skills.

Tests

  • New unit case in platform-guidance.test.ts: the path appears when skills are materialized and is absent otherwise. File passes; full runner suite 2117 passed; tsc --noEmit clean.
  • The dev runner picks this up via tsx watch. NOT yet verified in a live run; the What to QA step below is the live check.

What to QA

  • Create an agent with a skill attached, ask it something that makes it consult the skill. Its first read should hit the real path directly, with no ENOENT retry loop in the transcript.

…sing

Live session evidence (2026-08-10): asked about its skills, a model emitted
a fully-formed absolute skills path from pattern memory as its FIRST move —
structurally wrong (one id segment where the mount path has two) — got
ENOENT, then self-corrected by listing its real cwd, costing a failed call
and an approval interruption. The mount guidance names agent-files/ but
nothing named the skills read path.

New skills-read-path appendix states the harness-correct absolute path
(Pi: agents/skills snapshot; others: .{acpAgent}/skills), gated only on
materialized skills.
@dosubot dosubot Bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Aug 10, 2026
@vercel

vercel Bot commented Aug 10, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agenta-documentation Blocked Blocked Aug 10, 2026 2:12pm

Request Review

@dosubot dosubot Bot added the dev experience Improvement of the experience using the software. For instance better error messaging label Aug 10, 2026
@coderabbitai

coderabbitai Bot commented Aug 10, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Summary by CodeRabbit

  • New Features

    • Added clearer guidance for locating rendered skills in the sandbox environment.
    • Skill paths are shown only when skills are available, with instructions to use the provided path rather than reconstructing it.
    • Guidance now adapts to the active platform and keeps related instructions consistently ordered.
  • Bug Fixes

    • Improved reliability of skill-path instructions across supported agent environments.

Walkthrough

The sandbox environment now passes harness-specific rendered skill paths to platform guidance. The guidance tells agents to list the resolved directory and not construct its path. Unit tests cover conditional inclusion and existing guidance behavior.

Changes

Rendered skills path guidance

Layer / File(s) Summary
Add rendered skills path guidance
services/runner/src/engines/sandbox_agent/platform-guidance.ts
Platform guidance accepts an optional skills path and instructs agents to list the rendered directory instead of guessing its location.
Wire harness-specific skills paths
services/runner/src/engines/sandbox_agent/environment.ts
The environment selects agents/skills for Pi and .{acpAgent}/skills for other harnesses. It omits the path when no skills exist.
Validate prompt inclusion and ordering
services/runner/tests/unit/platform-guidance.test.ts
Tests verify conditional skills-path guidance and preserve coverage for tool gating, ordering, mount handling, instruction wording, and Codex-specific guidance.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related issues

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the main change: preventing models from guessing rendered-skills paths.
Description check ✅ Passed The description directly explains the rendered-skills path issue, the guidance change, and the related tests.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/skills-read-path-guidance

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@mmabrouk mmabrouk added the lgtm This PR has been approved by a maintainer label Aug 10, 2026
@mmabrouk
mmabrouk merged commit e6e999b into release/v0.112.0 Aug 10, 2026
39 of 42 checks passed

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
services/runner/tests/unit/platform-guidance.test.ts (1)

56-67: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Cover the independent skills-path gate.

run() supplies toolNames: ["commit_revision"] by default. A regression that adds the commit-tool gate to skillsPath would still pass this test. Add a case with skillsPath and toolNames: [].

Suggested regression test
     const withoutSkills = platformGuidanceAppendix(run({}));
     assert.ok(!withoutSkills?.includes("rendered skill files"));
+    const withoutCommitTool = platformGuidanceAppendix(
+      run({
+        skillsPath: "/tmp/agenta/mounts/p/s/agents/skills",
+        toolNames: [],
+      }),
+    );
+    assert.ok(
+      withoutCommitTool?.includes("/tmp/agenta/mounts/p/s/agents/skills"),
+    );

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: dd37c5d1-e838-4a3e-aa3b-a9502710c367

📥 Commits

Reviewing files that changed from the base of the PR and between df65653 and 3ea4238.

📒 Files selected for processing (3)
  • services/runner/src/engines/sandbox_agent/environment.ts
  • services/runner/src/engines/sandbox_agent/platform-guidance.ts
  • services/runner/tests/unit/platform-guidance.test.ts

@github-actions

Copy link
Copy Markdown
Contributor

Railway Preview Environment

Preview URL https://gateway-pr-5904.up.railway.app/w
Project agenta-oss-clone-spike
Image tag pr-5904-fc54d23
Status Deployed
Railway logs Open logs
Workflow logs View workflow run
Updated at 2026-08-10T15:22:07.327Z

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dev experience Improvement of the experience using the software. For instance better error messaging lgtm This PR has been approved by a maintainer size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant