Skip to content

Use byte offsets for YARA line lookup - #364

Open
rng1995 wants to merge 2 commits into
mainfrom
agent/yara-byte-offset-lines
Open

Use byte offsets for YARA line lookup#364
rng1995 wants to merge 2 commits into
mainfrom
agent/yara-byte-offset-lines

Conversation

@rng1995

@rng1995 rng1995 commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Jira

Jira: SKILLSPECT-12
Deliverable #3: Use byte offsets for YARA line lookup

Summary

  • Interpret yara-python match offsets as byte offsets against the exact UTF-8 buffer scanned by YARA.
  • Use byte-accurate line lookup for destructive/autonomy locality checks and reported finding locations.
  • Build finding context from the corrected 1-based line number so Unicode prefixes cannot shift the displayed evidence.

Problem

static_yara encodes skill content to UTF-8 before calling rules.match(data=...), so every StringMatchInstance.offset is a byte position. The analyzer previously passed those positions to helpers that sliced the original Python str as though they were character positions.

For content containing multibyte Unicode before a match, this could:

  1. report the wrong finding line and surrounding context; and
  2. distort the line distance between destructive and autonomy evidence, potentially turning distant evidence into a false HIGH finding or suppressing genuinely local evidence.

ASCII-only content masked the mismatch because its byte and character positions are identical.

Implementation

  • Add a dedicated byte-offset line helper that counts newlines in the scanned bytes buffer.
  • Pass that same buffer into the destructive/autonomy locality check.
  • Compute the finding's start_line from the YARA byte offset.
  • Derive context from the corrected line number instead of reusing a byte offset with a character-based helper.

Security invariants

  • No YARA rule content, conditions, severity, or confidence is changed.
  • The destructive/autonomy proximity limit remains three lines.
  • Unconditional rm -rf / blocking remains unchanged.
  • ASCII behavior is unchanged; the correction only removes byte/character ambiguity for multibyte input.

User impact

YARA findings now point to the correct line and context in files containing multibyte Unicode. The destructive/autonomy locality guard also remains accurate regardless of the characters preceding its evidence.

Validation

  • 62 passed in tests/nodes/analyzers/test_static_yara.py.
  • Added a regression proving a multibyte prefix preserves the exact finding line and context.\n- Added a multi-string regression proving a valid match at byte offset zero remains the first reported location.
  • Added a regression proving multibyte text cannot collapse distant destructive/autonomy evidence into a false match.
  • Ruff lint and format checks passed for both changed files.
  • git diff --check passed.

Files changed

  • src/skillspector/nodes/analyzers/static_yara.py
  • tests/nodes/analyzers/test_static_yara.py

Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
@rng1995
rng1995 force-pushed the agent/yara-byte-offset-lines branch from 5429c13 to 3545a85 Compare August 12, 2026 05:11
Comment thread src/skillspector/nodes/analyzers/static_yara.py
@rng1995
rng1995 marked this pull request as ready for review August 12, 2026 08:28
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
@rng1995
rng1995 force-pushed the agent/yara-byte-offset-lines branch from f5f9941 to b2fc637 Compare August 12, 2026 08:36
@rng1995
rng1995 requested a review from keshprad August 12, 2026 08:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant