Skip to content

feat(analyzer): detect external model selection - #426

Merged
rng1995 merged 2 commits into
NVIDIA:mainfrom
deepujain:feat/370-model-pin-cost-exposure
Aug 25, 2026
Merged

feat(analyzer): detect external model selection#426
rng1995 merged 2 commits into
NVIDIA:mainfrom
deepujain:feat/370-model-pin-cost-exposure

Conversation

@deepujain

Copy link
Copy Markdown
Contributor

Summary

  • add EA5 detection for external model or provider selection
  • report top-level SKILL.md frontmatter pins as MEDIUM cost exposure
  • report actionable coding-CLI shell-outs as HIGH silent account-switch exposure
  • avoid nested/frontmatter defaults, prose-only mentions, language-runtime -m flags, and unknown local model names

Validation

  • pytest tests/unit/test_patterns_new.py tests/nodes/analyzers/test_static_patterns.py -q (538 passed)
  • pytest -m 'not integration and not provider' tests/ -q (2818 passed, 13 skipped, 38 deselected, 4 xfailed)
  • ruff check src tests
  • ruff format --check src tests
  • real CLI JSON smoke: one HIGH command finding and one MEDIUM frontmatter finding

Risk

  • deterministic static analysis only; no provider calls or runtime model changes
  • generic model flags require a recognized model family and exclude Python/Node/Perl/Ruby module invocation
  • findings are advisory and describe billing/cost exposure rather than malicious intent

Fixes #370

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SkillSpector Review]

Requesting changes for a HIGH-severity prose false positive. Every inline-code command is classified as actionable regardless of the surrounding sentence, so documentation that merely names a supported CLI command is reported as silent model/account switching. Please require directive or shell-example context for inline code and add descriptive inline-code and fenced-example negative regressions.

@deepujain
deepujain force-pushed the feat/370-model-pin-cost-exposure branch 2 times, most recently from faf5bda to 69e8f78 Compare August 24, 2026 18:17
@rng1995

rng1995 commented Aug 24, 2026

Copy link
Copy Markdown
Collaborator

[SkillSpector Maintainer Ping]

@deepujain, thank you for the follow-up commit and thread reply. The new directive-context gate plus descriptive inline-code and fenced-example regressions appear to address the EA5 review feedback. Please resolve the review thread and reply here when you want the updated head re-reviewed.

The branch is currently behind main. GitHub reports it as mergeable with no conflicts, but please update it from main, resolve any conflicts that surface during that update, and keep the required checks green.

Signed-off-by: Deepak Jain <deepujain@gmail.com>
Signed-off-by: Deepak Jain <deepujain@gmail.com>
@deepujain
deepujain force-pushed the feat/370-model-pin-cost-exposure branch from fa0dbe6 to 052cd9f Compare August 24, 2026 18:50
@deepujain

Copy link
Copy Markdown
Contributor Author

Updated the branch onto current main at 052cd9f, resolved the addressed review thread, and reran the focused EA5/static-pattern suite (542 passed) plus repository-wide Ruff checks. The required GitHub checks are rerunning on the new head. Ready for re-review when convenient.

@mohgupta-ship-it

mohgupta-ship-it commented Aug 25, 2026

Copy link
Copy Markdown
Member

Powered by Codex: PR council review result.

This is a triage signal, not a maintainer approval.

  • Rating: new feature
  • Confidence: medium-high
  • Status read: Changes requested, green checks
  • Review method: fresh GitHub metadata/body/files/reviews/checks plus selected diffs; council lenses were spec fit, dead-code/reachability, YAGNI/scope, design/coupling, and code standards/tests.
  • Council assessment: External model-selection analyzer appears to address the earlier inline-prose false-positive path in sampled code/tests, but GitHub still records CHANGES_REQUESTED.
  • Recommended action: Request reviewer re-review before merge.

@rng1995
rng1995 merged commit e5440f6 into NVIDIA:main Aug 25, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature request: detect model/provider pins in skill frontmatter

3 participants