Skip to content

feat(ci): agent workflows for PR fixes, fork review, and queue triage - #172

Draft
defangdevs wants to merge 1 commit into
masterfrom
feat/self-maintaining-workflows
Draft

feat(ci): agent workflows for PR fixes, fork review, and queue triage#172
defangdevs wants to merge 1 commit into
masterfrom
feat/self-maintaining-workflows

Conversation

@defangdevs

Copy link
Copy Markdown
Owner

Three workflows toward a self-maintaining repo, split by trust boundary rather than by task. Draft: none of them can run until merged, and the secrets below are not set yet.

What each does

File Trigger Output
agent-pr.yml pull_request + agent-fix label, same-repo only Runs the flake checks relevant to the diff, regenerates modules/agent-box.nix on drift, fixes what is broken, pushes to the PR branch
agent-review-fork.yml pull_request_target, forks only Diff-only review comment. Does not run the checks
agent-triage.yml Weekly cron + workflow_dispatch Closes issues the code already fixed, labels the unlabeled, disposes of stale PRs, turns vague issues into actionable ones, flags dupes

Label-gating the PR agent is deliberate: each run costs API tokens and ~10 min of runner time on the nix store, so running on every synchronize would burn both on PRs that are already green.

Security effects

agent-review-fork.yml is the only workflow here reachable by an untrusted party. It needs pull_request_target because a fork's pull_request event gets no secrets and therefore no API key. Four properties keep that safe, and they are documented in the file header because breaking any one of them is how repos get compromised:

  1. Only the base ref is checked out — the PR head is never fetched and PR code never executes.
  2. permissions: is pull-requests: write and nothing else. No contents: write, no actions: write.
  3. The agent runs with --allowedTools "Read,Grep,Glob" and without --dangerously-skip-permissions. No Bash, no WebFetch — a diff instructing the model to exfiltrate $ANTHROPIC_API_KEY has no tool with which to do it. This is the opposite choice from agent-pr.yml, and the reason is that here the input is attacker-controlled.
  4. GH_TOKEN is absent from the agent step's environment; the diff is fetched before and the comment posted after.

The diff is wrapped in UNTRUSTED-DIFF markers with an instruction that injection attempts are findings to report, not instructions to follow.

agent-triage.yml holds contents: read, so it structurally cannot push code.

agent-pr.yml's github.actor != 'github-actions[bot]' guard is redundant todayGITHUB_TOKEN-authored events do not trigger workflows — and becomes the only thing preventing a self-review cascade the moment any job here moves to a PAT or GitHub App identity. That moment is coming: fixing the AMI publish gap needs an elevated identity. Keep the agent loop on GITHUB_TOKEN and give only the publisher the elevated one.

Why closing an issue requires two pieces of evidence

Triage rule 1 will not close an issue on a git log --grep match alone, because that match false-positives: grepping for #9 hits commits referencing #96 and #94. It must cite both a commit SHA and the behaviour's current location in the tree, or it drops to rule 4 and writes a concrete next step instead.

The rules have real debt to aim at right now: 7 of 29 open issues carry no label at all (#170, #167, #159, #154, #135, #127, #126), and two of three open PRs are parked [backup] drafts untouched since 28 Jul (#147, #148). Rule 3 comments on those and labels them stale but deliberately does not close them — you cannot know what a backup is protecting. #103 was the first rule-1 candidate and has already been closed by hand with its citation.

Checks run

All three files validated as parsing, with triggers, permissions, concurrency groups, the review matrix, and the job-level if: guards confirmed to resolve as intended. The workflows cannot execute until merged, and pull_request_target only ever runs the base-branch copy of its file — so behaviour is unverified until this lands. That is the main reason it is a draft.

No AWS cost, IAM, or networking impact.

Before merging — needs from a human

  • Set exactly one of CLAUDE_CODE_OAUTH_TOKEN (subscription auth, unmetered) or ANTHROPIC_API_KEY under Settings → Secrets and variables → Actions. Both set is an auth error.
  • Optionally CACHIX_AUTH_TOKEN + a CACHIX_CACHE repo var. The step skips cleanly without them, but with no warm store the PR agent spends most of its 30-minute budget rebuilding the VM closure rather than thinking. This is the single biggest determinant of whether agent-pr.yml is useful on this repo.
  • Confirm the @anthropic-ai/claude-code@2.1.0 pin is the version you want — pinned rather than @latest so the harness does not change under the job, but not verified against the registry.
  • Create the labels the workflows reference: agent-fix, agent-triaged, stale, needs-decision. None exist yet.

🤖 Generated with Claude Code

https://claude.ai/code/session_01PsqGhkpjsb4kKAdVz25xKN

Three GitHub Actions workflows that let the repo maintain itself, split by
trust boundary rather than by task:

- agent-pr.yml: own PRs, label-gated on `agent-fix`. Runs the flake checks
  relevant to the diff, regenerates modules/agent-box.nix on drift, fixes what
  is broken, pushes to the PR branch. Never touches master.
- agent-review-fork.yml: fork PRs, which cannot use the above because a fork's
  `pull_request` gets no secrets. Uses `pull_request_target` but never checks
  out the PR head, holds only `pull-requests: write`, and runs with an explicit
  `--allowedTools` allowlist and no `--dangerously-skip-permissions` — so a
  prompt injection in the diff has no tool with which to egress the API key.
  Comment only; the checks are not run.
- agent-triage.yml: weekly queue maintenance with five ordered rules, each of
  which removes a queue item or makes one routable. Closing an issue requires
  citing both the commit and the behaviour's current location — a `git log
  --grep` match alone false-positives (`#9` matches `#96`).

Security effects: agent-review-fork.yml is the only workflow here reachable by
an untrusted party, and the four properties that keep it safe are documented in
its header. agent-triage.yml holds `contents: read`, so it structurally cannot
push code. agent-pr.yml's `github.actor != 'github-actions[bot]'` guard is
redundant while everything runs on GITHUB_TOKEN, and becomes load-bearing the
moment any job here moves to a PAT or App identity.

No AWS cost, IAM, or networking impact. Checks run: all three files validated
as parsing, with triggers, permissions, concurrency groups, matrix and job
guards confirmed to resolve as intended. The workflows themselves cannot run
until merged (and `pull_request_target` only ever runs the base-branch copy).

Requires one of CLAUDE_CODE_OAUTH_TOKEN or ANTHROPIC_API_KEY in Actions
secrets; optionally CACHIX_AUTH_TOKEN + a CACHIX_CACHE var, without which the
PR agent spends most of its budget rebuilding the nix store.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PsqGhkpjsb4kKAdVz25xKN
@defangdevs

Copy link
Copy Markdown
Owner Author

Context and reasoning for this now live in #176 (self-maintaining repo tracker).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants