Skip to content

Fix Ruby analysis false positives and partial-fetch blindness - #80

Merged
braedonsaunders merged 1 commit into
mainfrom
fix/ruby-analysis-issues-76-79
Aug 6, 2026
Merged

Fix Ruby analysis false positives and partial-fetch blindness#80
braedonsaunders merged 1 commit into
mainfrom
fix/ruby-analysis-issues-76-79

Conversation

@braedonsaunders

Copy link
Copy Markdown
Owner

Fixes #76, fixes #77, fixes #78, fixes #79.

All four issues were filed from analyzing justi/ruby_llm-contract. This PR was validated by re-running the headless analyzer against a fresh clone of that repo before and after.

Results on the reporter's repo

Category Before After
Unused functions 342 (34/35 of reported ones false) 12 — each verified genuinely uncalled in-repo
Circular dependencies 19 (15 markdown pairs + 4 call-graph pairs, 0 import cycles) 0 (correct — the repo has no mutual require edges)
Test files found 0 90 — "Add Test Coverage" no longer fires
High complexity (lib/) contract.rb (40), eval_host.rb (40) only 11 files led by step/dsl.rb (93) and step/base.rb (86) — flog's top files now present and differentiated

The overarching root cause the issues couldn't see

The deployed site fetched only 50 of 226 files: unauthenticated GitHub API access allows 60 requests/hour and each file costs 2 (content + commits). Failed fetches were silently swallowed (.catch(function(){processFile(i+1);})), so the analysis ran on whatever happened to download before the limit hit — no spec/, and only 18 of 105 lib files. That single cause produced the "missing" top-complexity files in #78 and the plain-Constant.method-call misses in #76 (the callers were never fetched). Failed files now surface as a critical "Files Not Fetched — Partial Analysis" issue plus a notification.

(The "score saturates at 40" hypothesis in #78 was a coincidence: there is no cap; the two files genuinely both scored 40 under the old pattern set, which ignored elsif/unless/when/rescue/until.)

Changes

#76 — Ruby unused-function false positives

  • def self.foo extracts as foo, not self (the naming bug that hid the report's one true positive)
  • ?/! method names survive tokenization and the >250-name candidate filter
  • Parenless receiver calls (RubyLLM::Contract.eval_hosts) count as calls; bare non-definition occurrences of known method names count as usages — the same policy the Python tree-sitter path already uses, per the issue's "false negative is cheaper than a false positive" principle
  • initialize and runtime hooks (included, inherited, method_missing, …) are never reported dead
  • Unresolvable polymorphic calls mark every same-named definition possibly-called instead of being dropped

#77 — Circular dependencies

  • Cycles now require import statements in both directions, computed from the import graph — mutual markdown links and registry-callback call edges no longer qualify
  • Ruby require/require_relative edges are parsed and resolved (lib/ load-path convention), so genuine Ruby require cycles are still detectable

#78 — Complexity

  • Partial-fetch transparency (above)
  • Ruby branch keywords now count; the issue description names the metric (approximate cyclomatic complexity)

#79 — Test coverage

  • New shared Parser.isTestFile recognizing RSpec (spec/**/*_spec.rb), Minitest (test/**/*_test.rb), pytest (test_*.py), Go (*_test.go), JVM (*Test.java etc.), and the existing JS conventions; used by the coverage suggestion, the dead-function filter, and the architecture test classifier

Testing

  • node --test tests/*.test.mjs — 83/83 pass
  • tests/html-inline-script-analysis.smoke.js — pass
  • tests/codeflow-repo-smoke.mjs /tmp/ruby_llm-contract — 227/227 files, 0.3s
  • Headless before/after comparison on ruby_llm-contract as above; the 12 remaining unused functions were manually spot-checked (score_trend, drift? have zero references anywhere; estimate_eval_cost, minitest asserts are documented public API with no in-repo caller — the structural limit the reporter acknowledged in "unused functions": 34 of 35 entries on a Ruby gem are false positives, and the 1 real hit is reported as self #76 cause 6)

🤖 Generated with Claude Code

Validated against justi/ruby_llm-contract, the repo all four issues were
filed from. Unused-function false positives drop from 342 to 12 (the
remaining 12 are genuinely uncalled in-repo), circular dependencies drop
from 19 to 0, the test suite is now recognized, and the top complexity
files are no longer missing.

Issue #76 — "unused functions" on Ruby:
- def self.foo now extracts as "foo" instead of "self"
- Ruby ?/! method names survive tokenization and candidate filtering
- Receiver-dot calls without parens count as calls; any non-definition
  occurrence of a known method name counts as a usage (same policy as
  the Python tree-sitter path — a false negative is cheaper than a
  false positive in a dead-code report)
- initialize and runtime hooks (included/inherited/method_missing/...)
  are never reported dead
- When a call site cannot be resolved to a single definition, every
  same-named definition is marked possibly-called instead of dropping
  the call

Issue #77 — circular dependencies:
- Cycles now require actual import statements in BOTH directions,
  computed from the import graph instead of mutual call-graph or
  markdown-link edges. Mutual doc links and registry callbacks no
  longer count
- Ruby require/require_relative edges are now parsed and resolved
  (lib/ load-path convention, .rb extension)

Issue #78 — high complexity files:
- Root cause of the missing files: unauthenticated GitHub fetches that
  fail (60 req/hr limit) silently dropped files from the analysis. They
  now appear as a critical "Files Not Fetched — Partial Analysis" issue
  plus a notification instead of vanishing
- Ruby branch keywords (elsif/unless/when/rescue/until) now count, so
  Ruby files rank against each other sensibly
- The issue description now names the metric (approximate cyclomatic
  complexity)

Issue #79 — test coverage:
- Shared Parser.isTestFile recognizes spec/**/*_spec.rb, test/**/
  *_test.rb, test_*.py, *_test.go, *Test.java, __tests__, .test./.spec.
  and friends; used by the suggestion, the dead-function filter, and
  the architecture test classifier

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@vercel

vercel Bot commented Aug 6, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
codeflow Ready Ready Preview Aug 6, 2026 3:45pm

Request Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment