Walk declaration initializers scoped to the declared symbol — Kotlin, Java, TS/JS, Scala, Rust, Python (#693 for the other six languages) - #1511
Open
danusha2345 wants to merge 10 commits into
Conversation
added 10 commits
August 5, 2026 22:18
…lbymchenry#693 for Kotlin) The Kotlin property hook consumes the whole `property_declaration` subtree, so the dispatcher only scanned it for function-as-value candidates and never walked it for calls. Every call inside a property initializer was therefore dropped from the graph entirely — not misattributed, gone: private val fieldLambda: () -> Unit = { target() } // no caller private val samField = Runnable { target() } // no caller private val plain = compute() // no caller private val delegated by lazy { compute() } // no caller That is exactly how Android/MSDK callbacks are declared, so anything reached only through such a callback looked like it had no callers and its blast radius came back far too small. Fix: after minting the property node, walk its RHS — the named child after the `=` token plus a `property_delegate` — through visitFunctionBody with the property pushed on the scope stack. This is Go's colbymchenry#693/colbymchenry#744 fix ported; tree-sitter-kotlin exposes no fields at all, hence the `=` anchor instead of Go's `child_by_field_name("value")`. Not touched, by design: the `scope == "local"` early return, the hook-declined destructuring branch, and the declaration's own children (modifiers, `val`/`var`, name+type, extension receiver, `getter`/`setter`). Both arms move together — the Rust kernel and the TS extractor — so kernel-kotlin-parity stays byte-identical; the torture fixture gains the lambda/SAM/anonymous-object initializer shapes. Measured on a 113-file Android/Kotlin app: strictly additive, 0 lost nodes / edges / refs, +7 nodes (anonymous-object overrides that were invisible), +109 edges (+47 calls, +35 instantiates, +20 references), and the real case that motivated this — a `CameraFrameListener` field — now shows up as a caller of the method it invokes.
…ry#693 for Java) `extractField` minted the field node and stopped; the dispatcher then only scanned the `field_declaration` subtree for function-as-value candidates. So a field initializer's code was never walked: private final Runnable fieldLambda = () -> target(); // no caller private final Runnable l = new LocationListener() { … }; // invisible private final int eager = compute(); // no caller The anonymous-class form lost more than edges — the class and its overrides were never extracted at all, which is how `Parcelable.Creator` and every Android listener field is written. Fix: walk the declarator's `value` through visitFunctionBody with the field pushed on the scope stack — Go's colbymchenry#693/colbymchenry#744 fix, and the same shape as the TS/JS class-field walk already sitting in the methodTypes branch (colbymchenry#808). `extractField` is shared, but the walk is keyed on the `value` FIELD, which only Java's `variable_declarator` carries: C# (bare child), VB.NET (`initializer`) and PHP (`default_value`, separate branch) are untouched and stay for their own turn. Kernel and TS extractor move together, so kernel parity stays byte-identical. Measured on the 409-file DJI MSDK v5 UX SDK (Java): strictly additive, 0 lost nodes / edges / refs, +21 nodes (anonymous listener and Creator classes with their overrides), +21 edges, +651 refs, 409/409 files still byte-parity between the two arms.
…ymchenry#693 for TS/JS) Two defects in one place — extractVariable's JS-family branch. 1. The initializer walk ran with only the FILE on the node stack, so `const cfg = loadConfig()` recorded the FILE as loadConfig's caller. That is literally the leak colbymchenry#693 described and colbymchenry#744 fixed for Go; the closing note there said "JS/TS didn't have this gap", which holds only for the `const f = () => …` shape (an arrow value delegates to extractFunction and gets its own node). 2. Object literals were excluded from the walk outright, on the grounds that their function-valued members are extracted individually below — but that only happens for EXPORTED consts. `const obj = { handler: () => target() }` therefore contributed nothing at all: no member node, and no call edge to anything. Fix: walk the initializer with the declared symbol pushed on the stack, and skip only the shapes whose members really are extracted one-by-one (exported object-of-functions, RTK endpoints, Pinia setup, Vue store collections) — walking those too would double-count each member arrow's calls. Two follow-on repairs the change surfaced: - CFML `<cfscript>` bodies are delegated to a separate extractor as if they were a whole module, so a `var x = helper()` local inside a `<cffunction>` minted a top-level variable node and the walk attributed `helper` to it. Those snippet-top-level non-callables are locals of the enclosing function; their refs now redirect to it, like the snippet's file-attributed refs already did. - codegraph_explore stopped listing a synthesized dynamic-dispatch link once the same pair also had a static call edge: it asked getCallers/getCallees, which return one row per NEIGHBOUR (the colbymchenry#1086 de-dup), so the static edge hid the heuristic one. It now asks the edges directly. This is exactly the RTK thunk case — `dispatch(innerThunk(n))` inside a thunk initializer is now a real static edge, which is the point, but the synthesized hop must still show up in the summary. Measured on three independent TypeScript trees (codegraph's own src/, evcc-ng, gv-grx — 499 files): 0 lost nodes/edges/refs, 780 refs re-attributed from the file node to the declaring constant, 228 genuinely new refs from object literals, node and edge counts unchanged. Kernel parity holds (324/324 non-deferred files byte-identical).
Follow-up to cf582f4. `val url: String get() = build(host)` nests the accessor UNDER the property_declaration, so the hook consumed it and everything the getter body called vanished — the initializer walk deliberately skipped `getter`/`setter` on the theory that they are declaration, not code. They are code. Measured on the Android project this arc is validated against: 16 such properties, whose accessor bodies contributed nothing at all. An accessor written on its OWN line parses as a SIBLING of the property, not a child, so it stays out of reach here and keeps attributing to the enclosing class exactly as before. That asymmetry is the grammar's, and closing it means looking backwards from a sibling accessor to the preceding declaration — separate change, separate risk. Whole-project re-measure (110 Kotlin + 100 JS + 37 Go files, both arms of the extractor): 0 genuinely lost edges, +142 edges. The 13 edges that changed identity are 9 JS initializer calls moving off the file node onto their declaring constant, and 4 Kotlin `run(…)` call sites that used to name-match a JavaScript test helper in desktop/test/ and now match a Kotlin `run` override.
…olbymchenry#693 for Scala) The val/var hook minted the node and returned true, so the dispatcher only scanned the subtree for function-as-value candidates and the initializer's code was never walked: val fieldLambda: () => Unit = () => target() // no caller val direct = target() // no caller lazy val lazily = compute() // no caller val anon = new Runnable { def run() = target() } // no caller Scala puts almost everything in a val, so this is not an edge case: on a 32-file SpinalHDL project the graph was missing 812 references. Fix: walk the `value` field through visitFunctionBody with the declared symbol pushed on the scope stack — the same shape as Go's colbymchenry#693/colbymchenry#744 fix, which the grammar supports directly here (`val_definition` exposes `value`). Kernel and TS extractor move together; kernel parity holds. Measured on that SpinalHDL project: 0 lost refs, 0 re-attributed, +812 new, node and edge counts unchanged (824/792), 32/32 files byte-parity between the two arms.
…estructuring, own-line accessors Three shapes the hook still swallowed after cf582f4/8b2e436. The first two are outright losses: the code existed in the file and reached the graph nowhere. init { val cfg = load() } // `load` vanished — only the block's // bare statements survived val (a, b) = makePair() // `makePair` vanished, at class AND // file scope val x: Int // walked, but attributed to the get() = compute() // enclosing CLASS, not to `x` The first two mint no symbol of their own, so they are walked at the ENCLOSING scope — an `init` block's calls belong to the class, exactly like the block's other statements. The third is an accessor the grammar makes a following SIBLING of the property rather than a child (same-line accessors nest, which 8b2e436 already covered); the property now claims its own-line accessors, and the accessor branch skips what the property claimed. The skip re-derives the property's kind instead of remembering it across nodes, so a destructured or local declaration — which mints nothing and therefore claims nothing — keeps falling through as before. No cross-node state in either arm. The scope/kind decision moved into one function (`kotlinPropertyKind` / `property_kind`) now that two branches need it. Measured on the 113-file Android/Kotlin project, against upstream c65d56c: 0 lost refs, 0 re-attributed, +368 new. Kernel parity unchanged (99/113 byte-identical, same 14 grammar-deferred files as upstream).
…bol (colbymchenry#693 for Rust) `const_item`/`static_item` ride extractVariable's generic fallback, which minted the node and stopped — the initializer was never walked, so every call inside it was missing from the graph: const LEN: usize = compute_len(); // no caller static REGISTRY: Lazy<Cfg> = Lazy::new(|| build_cfg()); // no caller That second shape is how once_cell/lazy_static singletons, handler tables and static configs are written, so whatever they build looked unreferenced. Fix: walk the `value` field through visitFunctionBody with the declared symbol pushed on the scope stack. The fallback is shared, so the walk is gated to Rust — the other languages on it spell their initializer differently and get their own turn. The declared symbol is identified by the `name` FIELD, so the phantom node the fallback also mints for a bare-identifier initializer (`const MAX: u32 = OTHER;` → `MAX` plus a spurious `OTHER`) is left exactly as it is: a separate defect, deliberately not folded in. Measured on three Rust projects (gnss_rust, emmc-reader-gui, bot_predlogka_rust — 282 files): 0 lost refs, 0 re-attributed, +86 new; node and edge counts unchanged. Kernel parity holds (226/226 non-deferred files byte-identical).
…he name (colbymchenry#693 for Python) The assignment branch minted the node and stopped, so every call on the right-hand side was missing from the graph: APP = compute() # no caller handler = lambda: target() # no caller MAPPING = {"a": compute()} # no caller first, second = compute(), f() # no caller, and no symbol either That is everything a module wires up at import time — `app = FastAPI()`, `ENGINE = create_engine(url)`, `router = build_router()`, handler registries — so whatever those build looked unreferenced. Fix: walk the `right` field through visitFunctionBody with the assigned name pushed on the scope stack. A tuple target mints no symbol, so its RHS is walked at the enclosing scope rather than lost. Gated to Python — Ruby shares this branch and gets its own turn. Class attributes are untouched: they never reach this branch (the dispatcher's class-scope gate excludes them), so their calls keep riding the class node. Giving them symbols of their own is a separate question — there are no nodes for them today at all. A function-as-value in an initializer (`REGISTRY = {"org": Serializer}`) now produces a reference from BOTH the assigned name and the file node, because the dispatcher's own scan runs either way. That is the shape Go's colbymchenry#744 already ships — verified against the Go arm on the same input — so the two languages stay consistent; the function-ref test's expectation is updated to match. Measured on three Python trees (libresdr, ADRC-betaflight, bot_predlogka_big_project — 799 files): 0 lost refs, 0 re-attributed, +3057 new; node and edge counts unchanged. Kernel parity holds — 6221/6221 files byte-identical across the sweep.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1510. Ports #693's fix (merged as #744 for Go) to Kotlin, Java, TypeScript/JavaScript, Scala, Rust and Python.
The shape is the same everywhere: after minting the declaration's node, walk its initializer through
visitFunctionBodywith that node pushed on the scope stack, so the calls inside it are attributed to the symbol that owns them instead of leaking to the file node — or, in most of these languages, vanishing outright.Every language moves in both arms together —
src/extraction/and the matchingcodegraph-kernel/src/walker — sokernel-*-paritystays byte-identical; that is asserted per language, and each torture fixture gains the new shapes.Per language
Kotlin — three commits. The
property_declarationhook consumed the whole subtree, so the dispatcher only fn-ref-scanned it.= …) and aproperty_delegate(by lazy { … }) are now walked under the property.val x: Int get() = f()nests under the declaration, an own-line one parses as a following sibling. The property claims its own-line accessors and the accessor branch skips what the property claimed — the skip re-derives the property's kind instead of remembering it across nodes, so a destructured or local declaration (which mints nothing and therefore claims nothing) keeps falling through. No cross-node state in either arm.init { val cfg = load() }andval (a, b) = makePair()mint no symbol of their own, so they are walked at the enclosing scope — an init block's calls belong to the class, like its other statements. Both used to disappear completely.kotlinPropertyKind/property_kind) now that two branches need it.Java —
extractFieldminted the node and stopped. The declarator'svalueis now walked under the field. The anonymous-class form lost more than edges:new LocationListener() { … }in a field initializer was never extracted at all, so the class and its overrides did not exist as symbols.extractFieldis shared, but the walk is keyed on thevaluefield, which only Java'svariable_declaratorcarries — C# (bare child), VB.NET (initializer) and PHP (default_value, separate branch) are untouched.TypeScript / JavaScript — two defects in
extractVariable's JS-family branch.nodeStack, soconst cfg = loadConfig()recorded the file as loadConfig's caller. (This is the part CodeGraph does not index call edges from anonymous/lambda functions #693's closing note assumed JS/TS didn't have; it holds only forconst f = () => …, where the arrow value delegates toextractFunction.)const obj = { handler: () => target() }therefore contributed nothing at all.Both fixed by walking the initializer scoped to the declared symbol and skipping only the shapes whose members really are extracted one-by-one (exported object-of-functions, RTK endpoints, Pinia setup, Vue store collections) — walking those too would double-count each member arrow's calls.
Scala — the
val/varhook returnedtruewithout walking; the grammar exposesvaluedirectly, so this is the closest analogue to the Go fix. Scala puts almost everything in aval, which is why it shows the largest per-file gain here.Rust —
const_item/static_itemrideextractVariable's generic fallback. That fallback is shared, so the walk is gated to Rust; the declared symbol is identified by thenamefield, which leaves the fallback's phantom node for a bare-identifier initializer (const MAX: u32 = OTHER;→MAXplus a spuriousOTHER) exactly as it is. Separate defect, deliberately not folded in — noted in #1510.Python — the
assignmentbranch minted the node and stopped.rightis now walked under the assigned name; a tuple target mints no symbol, so its RHS is walked at the enclosing scope rather than lost. Gated to Python — Ruby shares the branch. Class attributes are untouched: they never reach this branch (the dispatcher's class-scope gate excludes them), so their calls keep riding the class node; giving them symbols of their own is a separate question, since there are no nodes for them today at all.Two follow-on repairs the change surfaced
Both are in this PR because the initializer walk is what exposed them.
CFML.
<cfscript>bodies are delegated to a separate extractor as if they were a whole module, so avar x = helper()local inside a<cffunction>minted a top-level variable node and the walk attributedhelperto it. Those snippet-top-level non-callables are locals of the enclosing function; their refs now redirect to it, exactly like the snippet's file-attributed refs already did.codegraph_exploredynamic-dispatch links. The section stopped listing a synthesized edge once the same pair also had a static call edge: it askedgetCallers/getCallees, which return one row per neighbour (the #1086 de-dup), so the static edge hid the heuristic one. It now asks the edges directly. This is the RTK thunk case —dispatch(innerThunk(n))inside a thunk initializer is now a real static edge, which is the point, but the synthesized hop must still show up in the summary.Validation
Each language was measured by diffing one extractor arm against upstream
mainover real trees, classifying every ref change as lost / re-attributed (same file+kind+name+line, different owner) / new:Node and edge counts are unchanged everywhere except Java (+21 nodes — the anonymous listener and
Parcelable.Creatorclasses that were previously invisible).End-to-end on a mixed 110-Kotlin / 100-JS / 37-Go project, full
codegraph initboth sides: 4311 → 4318 nodes, 12106 → 12254 edges, 0 edges lost. 13 edges changed identity — 9 JS initializer calls moving off the file node onto their declaring constant, and 4 Kotlinrun(…)call sites that used to name-match a JavaScript test helper indesktop/test/and now match a Kotlinrunoverride.The headline case that started this: an MSDK
ICameraStreamManager.CameraFrameListenerfield now appears incallersof the method it invokes.npm test— 2930 passing, on top of upstreammainas of a7db24d.Test coverage added
__tests__/extraction.test.ts— a regression test per language (Kotlin ×3 shapes, Java, TS/JS, Scala, Rust, Python), each asserting the exact set of owners for the calls in question, including the cases that deliberately still ride the enclosing scope.__tests__/function-ref.test.ts— one expectation updated: a function-as-value in an initializer (REGISTRY = {"org": Serializer}) now produces a reference from both the assigned name and the file node, because the dispatcher's own scan runs either way. That is the shape fix(go): attribute calls inside top-level closures to the var, not the file (#693) #744 already ships — verified against the Go arm on the same input — so the two languages stay consistent.Deliberately not in this PR
valuefield) and, more importantly,extractPropertywalks neither the accessor bodies nor=> exprnor{ get; } = …. Bigger than a field initializer; described in Declaration initializers are still unwalked (or unscoped) outside Go — #693 was fixed for Go only #1510.object→INSTANCEgap reported under Tracking: chained factory/singleton call resolution across statically-typed languages #750 is not touched here.