Skip to content

admin-bypass-sweep (4) an unanswered scope question resolves narrow - #504

Open
EdbertChan wants to merge 4 commits into
mainfrom
plan/admin-bypass-sweep-4-an-unanswered-scope-question-resolves-narrow
Open

admin-bypass-sweep (4) an unanswered scope question resolves narrow#504
EdbertChan wants to merge 4 commits into
mainfrom
plan/admin-bypass-sweep-4-an-unanswered-scope-question-resolves-narrow

Conversation

@EdbertChan

@EdbertChan EdbertChan commented Sep 12, 2026

Copy link
Copy Markdown
Owner

Summary

One skill helps an agent merge many pull requests past the normal checks. First it must ask how much to merge, and get a fixed yes sentence.

The problem: in a real run, both questions went out in one message. The person typed only the yes sentence and said nothing about how much.

The agent read that silence as a yes to the bigger choice. It then rewrote five branches by force and merged two linked groups.

Both planned checks held. The gap was a third path: filling in the missing answer with the largest option.

The fix asks the two questions in separate turns. If the "how much" question gets no clear answer, the agent does only the smallest choice it offered.

Review Claim

When the person answers only the yes sentence, the merge skill does the smallest job it offered, never the biggest.

Review Lane

behavior

Review Unit

product-skill

Safety Invariant

A sweep of single-item groups reads the same as before; only a multi-item group needs the second answer, and an unclear offer resolves to the smallest one named.

Slice Rationale

This is one step of the sweep skill's intake: Step 2, where scope and consent are asked. Other intake steps ship as their own slices.

Non-goals

  • Does not change the wording of the consent sentence.
  • Does not add any other way to approve.
  • Does not change Step 4's merge commands.
  • Does not touch any other skill.

Test Plan

Test Plan
  • python3 -m unittest discover -s product/skills/admin-bypass-sweep/tests -v
test_consent_only_reply_resolves_to_narrowest_scope (test_scope_intake.TestScopeIntake.test_consent_only_reply_resolves_to_narrowest_scope) ... ok
test_scope_answered_fixture_resolves_to_stated_scope (test_scope_intake.TestScopeIntake.test_scope_answered_fixture_resolves_to_stated_scope) ... ok
test_step_2_separates_scope_from_consent (test_scope_intake.TestScopeIntake.test_step_2_separates_scope_from_consent) ... ok

----------------------------------------------------------------------
Ran 3 tests in 0.000s

OK
  • python3 engine/skills/make-pr/scripts/preflight.py --base main
unit    product-skill: 4 file(s)
declare Review Unit: product-skill
        ok      codify-has-code
        ok      no dated provenance
        ok      skills three-harness check
        ok      ecosystem boundaries
        ok      skill file refs
        ok      skill test coverage
        ok      skill trigger mechanism
        ok      skill trigger policy
        ok      subagent scope contract
        ok      all 17 scenario(s) behaved as declared
ok      preflight passed
  • bash scripts/scrub-handoff-artifacts.sh
scrub-handoff-artifacts-ok

Fixtures added under product/skills/admin-bypass-sweep/tests/:

  • scope_unanswered_consent_only_reply.md: the reply has only the consent sentence, so the fixture expects single-PR groups only.
  • scope_answered_reply.md: scope is answered first ("Run the full grouped plan."), then consent comes in a separate turn, so the fixture expects the full plan.
  • test_scope_intake.py: checks that Step 2 holds the separate-turn text and the narrowest-scope text, and that both fixtures state their expected result.

Revert Plan

Revert Plan
  • Safe to revert? Yes
  • Revert command: git revert <merge-sha>
  • Post-revert steps: None. Reverting brings back the old Step 2 text, which does not say what to do with a reply that has only the consent sentence.
  • Data migration? No

🤖 Generated with Claude Code


Note

Low Risk
Documentation and test harness for agent intake behavior only; no merge commands or runtime code paths change.

Overview
Step 2 intake for admin-bypass-sweep now splits scope from consent and defaults unclear scope to the smallest offered option.

The skill text no longer bundles scope confirmation with a vague reconfirm. The agent must ask only the scope question in one turn (narrowest choice first, then named stacks or the full plan), then request the STOP literal consent sentence in a later turn. If the human replies with only the consent sentence, that counts as authorization but not as choosing scope—execution stays at single-PR groups only; multi-PR stacks need an explicit scope answer naming them or a larger option.

Two markdown fixtures document the answered-scope vs consent-only paths, and test_scope_intake.py locks in the Step 2 wording and fixture expectations. Merge steps (Step 4) and the consent sentence itself are unchanged.

Reviewed by Cursor Bugbot for commit 9e2ef67. Bugbot is set up for automated code reviews on this repo. Configure here.

Invoker Bot and others added 4 commits September 12, 2026 19:12
…cope question and the consent sentence in separate turns, and resolve silence to the narrower scope.

Review claim: An unanswered scope question resolves to the narrower option, never the larger one.
Review lane: behavior
Safety invariant: A sweep of single-item groups reads the same as before; only a multi-item group needs the second answer, and an unclear offer resolves to the smallest one named.
Effectiveness measurement: A fixture transcript answering only the consent sentence must resolve to singles-only, and one answering both must resolve to the stated scope. Both asserted in the skill's test directory.
Slice rationale: One step of the sweep skill's intake procedure; stacked behind the diff-read step because both edit the same SKILL.md.
Architectural effect: The written intake no longer contains a state where an unanswered question resolves upward.
Goal: Change Step 2 of product/skills/admin-bypass-sweep/SKILL.md so the scope question and the consent sentence are asked in separate turns, and an unanswered scope question resolves to the narrower option offered.
Motivation: A sweep asked for the consent sentence and the scope question in one message. The human's entire reply was the consent sentence. The agent recorded this as consent for the full 19-PR plan and proceeded, including five force-pushed rebases.
Alternative considerations: Re-asking after proceeding was rejected because the merges are not reversible. Treating the consent sentence as scope-bearing was rejected because the skill forbids paraphrase of it, so it cannot carry variable content.
Implementation details: Rewrite Step 2 so it asks in two turns: first the grouped plan plus the scope question, then, after a scope answer, the consent sentence request. State that a reply carrying only the consent sentence resolves scope to the narrowest option that was offered, and that multi-PR stacks require an explicit second answer naming them. Add fixtures under the skill's tests directory for a scope-answered and a scope-unanswered transcript.
Non-goals: Does not change the consent sentence's wording, does not introduce another way to approve, does not alter Step 4's merge commands, and does not touch any other skill.
Layer: domain
Feature state: active
Files: product/skills/admin-bypass-sweep/SKILL.md, product/skills/admin-bypass-sweep/tests/
Change types:
- product/skills/admin-bypass-sweep/SKILL.md: docs-only
- product/skills/admin-bypass-sweep/tests/fires_scope_unanswered.md: create
- product/skills/admin-bypass-sweep/tests/stays_silent_scope_answered.md: create
Acceptance criteria:
- `python3 -m unittest discover -s product/skills/admin-bypass-sweep/tests -v` exits 0.
- Step 2 asks scope and consent in separate turns.
- A reply carrying only the consent sentence is documented as resolving to the narrowest offered scope.

Solution:
  Ask the scope question and the consent sentence in separate turns, and resolve silence to the narrower scope.
Review claim: An unanswered scope question resolves to the narrower option, never the larger one.
Review lane: behavior
Safety invariant: A sweep of single-item groups reads the same as before; only a multi-item group needs the second answer, and an unclear offer resolves to the smallest one named.
Effectiveness measurement: A fixture transcript answering only the consent sentence must resolve to singles-only, and one answering both must resolve to the stated scope. Both asserted in the skill's test directory.
Slice rationale: One step of the sweep skill's intake procedure; stacked behind the diff-read step because both edit the same SKILL.md.
Architectural effect: The written intake no longer contains a state where an unanswered question resolves upward.
Goal: Change Step 2 of product/skills/admin-bypass-sweep/SKILL.md so the scope question and the consent sentence are asked in separate turns, and an unanswered scope question resolves to the narrower option offered.
Motivation: A sweep asked for the consent sentence and the scope question in one message. The human's entire reply was the consent sentence. The agent recorded this as consent for the full 19-PR plan and proceeded, including five force-pushed rebases.
Alternative considerations: Re-asking after proceeding was rejected because the merges are not reversible. Treating the consent sentence as scope-bearing was rejected because the skill forbids paraphrase of it, so it cannot carry variable content.
Implementation details: Rewrite Step 2 so it asks in two turns: first the grouped plan plus the scope question, then, after a scope answer, the consent sentence request. State that a reply carrying only the consent sentence resolves scope to the narrowest option that was offered, and that multi-PR stacks require an explicit second answer naming them. Add fixtures under the skill's tests directory for a scope-answered and a scope-unanswered transcript.
Non-goals: Does not change the consent sentence's wording, does not introduce another way to approve, does not alter Step 4's merge commands, and does not touch any other skill.
Layer: domain
Feature state: active
Files: product/skills/admin-bypass-sweep/SKILL.md, product/skills/admin-bypass-sweep/tests/
Change types:
- product/skills/admin-bypass-sweep/SKILL.md: docs-only
- product/skills/admin-bypass-sweep/tests/fires_scope_unanswered.md: create
- product/skills/admin-bypass-sweep/tests/stays_silent_scope_answered.md: create
Acceptance criteria:
- `python3 -m unittest discover -s product/skills/admin-bypass-sweep/tests -v` exits 0.
- Step 2 asks scope and consent in separate turns.
- A reply carrying only the consent sentence is documented as resolving to the narrowest offered scope.

Invoker-Finalize-Id: 26704a9f-70f5-4458-baa2-b9bd9a3ffe15
…rministic proof for the narrow scope default.

Review claim: The skill's fixtures assert silence on scope resolves narrow.
Review lane: proof
Safety invariant: Proof-only; adds no product behavior.
Effectiveness measurement: The suite fails if the scope-unanswered fixture resolves to the full sweep.
Slice rationale: One proof slice for this step.
Architectural effect: None; verification only.
Goal: Prove the narrow scope default deterministically.
Motivation: The defect is an inferred approval, so the proof asserts the inference is refused.
Alternative considerations: Manual verification was rejected as non-deterministic.
Implementation details: Execute the command below as the terminal proof.
Non-goals: No product edits here; proof only.
Layer: app_regression
Feature state: active

Exit code: 0
Invoker-Finalize-Id: 100703ed-6845-4470-b569-241d7f980dde
… that no ephemeral inter-task handoff files remain.

Review claim: The workflow leaves no scratch handoff artifacts behind.
Review lane: cleanup
Safety invariant: Read-only; never deletes files, alters the index, or commits caller work.
Effectiveness measurement: The gate fails when a plans/invoker-handoff.* or lens-*.json file is still present after the leaf tasks complete.
Slice rationale: One terminal hygiene gate for the workflow.
Architectural effect: None; check only.
Goal: Confirm no ephemeral handoff files survive the run.
Motivation: Inter-task scratch files leak into diffs and read as part of the work.
Alternative considerations: Deleting them automatically was rejected; the gate reports, it does not mutate.
Implementation details: Run scripts/scrub-handoff-artifacts.sh without --apply.
Non-goals: No deletion, no index changes, no commits.
Layer: app_regression
Feature state: active

Exit code: 0
…aa912f366-cf32078b — Terminal check that no ephemeral inter-task handoff files remain.

Review claim: The workflow leaves no scratch handoff artifacts behind.
Review lane: cleanup
Safety invariant: Read-only; never deletes files, alters the index, or commits caller work.
Effectiveness measurement: The gate fails when a plans/invoker-handoff.* or lens-*.json file is still present after the leaf tasks complete.
Slice rationale: One terminal hygiene gate for the workflow.
Architectural effect: None; check only.
Goal: Confirm no ephemeral handoff files survive the run.
Motivation: Inter-task scratch files leak into diffs and read as part of the work.
Alternative considerations: Deleting them automatically was rejected; the gate reports, it does not mutate.
Implementation details: Run scripts/scrub-handoff-artifacts.sh without --apply.
Non-goals: No deletion, no index changes, no commits.
Layer: app_regression
Feature state: active
@cursor

cursor Bot commented Sep 12, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_9af8d706-d101-4092-9463-2ac70c282d88)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant