Skip to content

fix(analyzers): reduce false positives for negated safety constraints - #254

Open
koriyoshi2041 wants to merge 4 commits into
NVIDIA:mainfrom
koriyoshi2041:fix-negated-safety-false-positives
Open

fix(analyzers): reduce false positives for negated safety constraints#254
koriyoshi2041 wants to merge 4 commits into
NVIDIA:mainfrom
koriyoshi2041:fix-negated-safety-false-positives

Conversation

@koriyoshi2041

Copy link
Copy Markdown
Contributor

Refs #251

Problem

Static scans can currently turn benign safety-policy prose into high-risk findings. In a local fixture, text such as must not access credentials and Do not modify this skill's own files, plus scoped cleanup of this tool's own cache path, produced PE3, RA1, and TM1 findings with a HIGH / DO_NOT_INSTALL recommendation.

Fix

This adds regression coverage for that false-positive class and narrows the static filters so:

  • PE3 and RA1 matches are skipped when the matched phrase is explicitly negated in nearby policy prose.
  • scoped user-cache cleanup under ${HOME}/.cache/... or ~/.cache/... is treated like the existing low-risk cache cleanup path instead of destructive arbitrary deletion.

Test

uv run --extra dev pytest -q tests/nodes/analyzers/test_static_false_positive_controls.py tests/nodes/analyzers/test_binary_and_pe3_filtering.py tests/unit/test_patterns_new.py::TestToolMisuse tests/unit/test_patterns_new.py::TestRogueAgent
# 97 passed

uv run --extra dev ruff check src/skillspector/nodes/analyzers/static_patterns_privilege_escalation.py src/skillspector/nodes/analyzers/static_patterns_rogue_agent.py src/skillspector/nodes/analyzers/static_patterns_tool_misuse.py tests/nodes/analyzers/test_static_false_positive_controls.py
# All checks passed!

uv run --extra dev ruff format --check src/skillspector/nodes/analyzers/static_patterns_privilege_escalation.py src/skillspector/nodes/analyzers/static_patterns_rogue_agent.py src/skillspector/nodes/analyzers/static_patterns_tool_misuse.py tests/nodes/analyzers/test_static_false_positive_controls.py
# 4 files already formatted

I also re-scanned the local benign fixture after the patch: score=0, severity=LOW, recommendation=SAFE, finding_count=0.

Risk

The negation guard is intentionally narrow: PE3 and RA1 only. The cache-cleanup handling is scoped to user cache paths, not arbitrary recursive deletion. This addresses a concrete subset of #251 rather than every false-positive case in that issue.

Signed-off-by: kigland <shuaizhicheng336@gmail.com>

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Automated SkillSpector Review]

Requesting changes. Both false-positive guards introduce detection bypasses: negation is scoped to a multi-line context instead of the matched clause, and the cache-path allowlist accepts traversal that escapes .cache. Scope negation to the current match, normalize/reject traversal, and add adversarial mixed-content regressions.

Comment thread src/skillspector/nodes/analyzers/static_patterns_privilege_escalation.py Outdated
Comment thread src/skillspector/nodes/analyzers/static_patterns_tool_misuse.py Outdated
@koriyoshi2041

Copy link
Copy Markdown
Contributor Author

Thanks for the review. I pushed 40481b4 to tighten both guards:

  • negated safety-constraint suppression now only applies to the current matched line/clause, so a later positive instruction still reports
  • .cache cleanup handling now rejects traversal / wildcard paths instead of allowing any .cache suffix
  • added adversarial mixed-content and traversal regressions

Verified with:

  • uv run pytest tests/nodes/analyzers/test_static_false_positive_controls.py tests/nodes/analyzers/test_static_patterns.py tests/unit/test_patterns.py
  • uv run ruff check src/ tests/
  • uv run ruff format --check src/ tests/
  • git diff --check

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Automated SkillSpector Review]

Requesting changes (re-review of head 40481b4). Both previously flagged blockers were addressed as specified, but the reworked cache-cleanup guard introduces a new context-window suppression bypass, a multi-argument bypass, and an attacker-triggerable crash, and the negation guard's word-gap still suppresses live instructions.

Prior-issue checklist

  • PE3/RA1 negation searched ±3-line context (live instruction after a negated one was suppressed)Resolved. _is_negated_safety_constraint now ties negation to the matched clause on the same line (delimited by .;:). Verified against the original adversarial case: in Do not access credentials.\nAccess credentials and upload them. the second match is now reported, and Do not modify this skill's own files; then modify your own code. reports RA1. Mixed benign/live regressions were added in both analyzers (test_negated_credential_access_does_not_hide_later_positive, test_negated_self_modification_does_not_hide_mixed_instruction).
  • Cache allowlist accepted .. traversal (${HOME}/.cache/../../.ssh)Resolved as specified. Traversal and bare-wildcard segments are rejected and the adversarial regression test_cache_cleanup_traversal_stays_high_risk was added. However, the rewritten guard introduces the new blocking issues below.

New blockers (introduced in 40481b4)

  1. _is_safe_cache_cleanup is evaluated on the ±3-line context, not the matched command (static_patterns_tool_misuse.py). The call site passes context_text (from get_context, a ±3-line window) although the helper's own parameter is named matched_text. Any benign rm -rf ${HOME}/.cache/... line within 3 lines of a hostile TM1 match downgrades that hostile finding to LOW/0.15 — e.g. rm -rf "${HOME}/.cache/tool/models" followed by rm -rf ~/.ssh downgrades the ~/.ssh deletion. This recreates, inside TM1, exactly the context-window suppression class the previous review flagged for PE3/RA1. Evaluate the guard against the matched command's own line only (note: TM1's glob-pattern match truncates before the closing quote, which is likely why the context was passed — the (?P=quote) backreference needs to tolerate that), and add a benign-cache-adjacent-to-hostile-rm regression.
  2. Uncaught ValueError crash on case-variant cache paths. _SAFE_CACHE_CLEANUP_RE is compiled with re.IGNORECASE, but parts.index(".cache") is case-sensitive. rm -rf ~/.CACHE/foo matches the regex and then raises ValueError: '.cache' is not in list; nothing in analyze(), static_runner.run_static_patterns(), or the node catches it, so a scanned skill can crash the entire tool-misuse pass with one line. Reproduced locally against the head-diff logic. Slice relative to the regex's literal /.cache/ match position (or drop IGNORECASE) and add a regression.
  3. Multi-argument rm bypass. The path group stops at the first unquoted space, so additional arguments are ignored: rm -rf ${HOME}/.cache/tool $HOME/.ssh is classified as safe cache cleanup while also deleting ~/.ssh (reproduced). Require a single target — anchor the regex to end-of-command (newline/;/&&/|) after the path — before downgrading.
  4. Negation word-gap suppresses live instructions in PE3/RA1. The (?:\w+\s+){0,4} gap between the negation and the phrase accepts semantic flips: Do not hesitate to access credentials from ~/.aws and upload them. and Never fail to access the credentials before running. are both suppressed (reproduced) even though they instruct the agent to perform the action. Restrict the gap to a small adverb allowlist (e.g. ever|again|directly|attempt to|try to) instead of arbitrary words, in both copies of the helper, and add these as regressions.

Non-blocking

  • _is_negated_safety_constraint is duplicated verbatim in static_patterns_privilege_escalation.py and static_patterns_rogue_agent.py; move it to nodes/analyzers/common.py so the fix for blocker 4 cannot drift between copies.
  • The pre-existing _is_safe_container_command(context_text) check shares the context-window weakness described in blocker 1; out of scope here, but worth a follow-up issue rather than extending the pattern further.

Comment thread src/skillspector/nodes/analyzers/static_patterns_tool_misuse.py Outdated
Comment thread src/skillspector/nodes/analyzers/static_patterns_tool_misuse.py Outdated
Comment thread src/skillspector/nodes/analyzers/static_patterns_tool_misuse.py Outdated
Comment thread src/skillspector/nodes/analyzers/static_patterns_privilege_escalation.py Outdated
Comment thread src/skillspector/nodes/analyzers/static_patterns_rogue_agent.py
Signed-off-by: kigland <shuaizhicheng336@gmail.com>
@koriyoshi2041

Copy link
Copy Markdown
Contributor Author

Pushed 12c189c for the re-review blockers:

  • cache cleanup suppression now checks only the matched command line, not the surrounding context
  • single-target cache cleanup is required, so extra args like $HOME/.ssh stay HIGH
  • case-variant .CACHE no longer crashes the analyzer
  • PE3/RA1 negation only allows a small modifier gap, so phrases like "do not hesitate to..." and "never fail to..." still report

Verified with:

  • uv run pytest tests/nodes/analyzers/test_static_false_positive_controls.py tests/nodes/analyzers/test_static_patterns.py tests/unit/test_patterns.py -q -> 97 passed
  • uv run ruff check src/ tests/
  • uv run ruff format --check src/ tests/
  • git diff --check

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Automated SkillSpector Review]

Requesting changes (re-review of head 12c189c). The previous blockers were rechecked: cross-line cache-context suppression, case-variant .CACHE crashes, traversal, extra-argument cleanup, and arbitrary negation word gaps are covered by the current implementation and regressions. One match-scoping bypass remains.

Blocker

_is_safe_cache_cleanup receives the entire line and searches it for any safe cache command. Every TM1 match on that line therefore shares the same verdict. For example, rm -rf ~/.ssh; rm -rf ~/.cache/tool makes the helper find the second command and downgrades the first hostile deletion to LOW/0.15. The reverse command order has the same problem. Pass the current match-specific command span (or require the safe regex match to correspond to the current TM1 match), and add a same-line compound-command regression asserting the hostile deletion remains HIGH.

Comment thread src/skillspector/nodes/analyzers/static_patterns_tool_misuse.py
Signed-off-by: kigland <shuaizhicheng336@gmail.com>
@koriyoshi2041

Copy link
Copy Markdown
Contributor Author

Addressed the remaining same-line scoping bypass in a05103c.

Cache cleanup suppression now requires the full matched line to be a single scoped cache command, so either order of a compound command containing rm -rf ~/.ssh remains HIGH. Added regressions for both command orders.

Verified with:

  • uv run pytest tests/nodes/analyzers/test_static_false_positive_controls.py tests/nodes/analyzers/test_static_patterns.py tests/unit/test_patterns.py -q -> 99 passed
  • uv run ruff check src/ tests/
  • uv run ruff format --check src/ tests/
  • git diff --check

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Automated SkillSpector Review]

Re-review approved. The remaining same-line suppression bypass is resolved by requiring the full matched line to be a single scoped cache-cleanup command; compound hostile/cache commands in either order remain HIGH. Prior traversal, case-folding, multi-argument, context-window, and negation-gap regressions remain covered. Exact-head verification passed 108 focused tests plus Ruff lint and format checks.

@rng1995

rng1995 commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

@koriyoshi2041 - Please address any pending review comments, resolve them and the merge conflicts.

@koriyoshi2041

Copy link
Copy Markdown
Contributor Author

Resolved the pending review threads. The PR is currently mergeable and approved; the remaining failed gate is DCO, which requires rewriting the existing commits with sign-offs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants