From 07e1f1d0315311d3a6c28cc255d2ace423fa622e Mon Sep 17 00:00:00 2001 From: Cryptskii <47649969+cryptskii@users.noreply.github.com> Date: Fri, 14 Aug 2026 18:16:15 -0400 Subject: [PATCH 1/6] ci: restore CodeQL security analysis, which has been off since 2026-03-26 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A workflow named "CodeQL" has run green on this repository every day for five months. It is not the security scan. Its init step reports: analysis-kinds: code-quality disable-default-queries: true - uses: code-quality That is GitHub's code quality default setup — the maintainability suite over javascript-typescript and python, uploading `*.quality.sarif`. It emits no security results. Meanwhile `code-scanning/default-setup` reports `state: not-configured`: the security half was switched off and the quality half kept running under the same name, so the loss of coverage looked exactly like a passing check. The last recorded security analysis is 2026-03-26: /language:rust 178 results -> 100 alerts /language:actions 4 results /language:javascript-typescript 4 results /language:python 0 results /language:c-cpp 0 results All 100 rust alerts were triaged on 2026-03-26 and 2026-03-29 (50 "false positive", 50 "won't fix"). Dismissals are keyed by alert fingerprint and survive re-enablement, so this change does not resurrect them — but the current "0 open alerts" reflects that triage, not five clean months. Nothing merged since has been seen by a security query: the FORS address collision, the forged-signature verifier, the domain-separation cut, the Kyber binding enforcement, the BLE trust-root work. Restore it as advanced setup so it cannot be switched off without a diff: - rust, javascript-typescript, python, actions — the pre-2026-03-26 set. - java-kotlin — new. It was never in that matrix, so the 85-file Android client has never been analyzed at all. - c-cpp — deliberately not restored. It returned 0 results because there is no first-party C to extract (3 headers, no .c/.cpp), and it cannot run build-mode: none. build-mode: none throughout. Requiring a ~30-minute --all-features workspace build is how Rust coverage gets dropped again the next time the build is slow. No `queries:` override: same default suite as before, so the alert delta stays attributable to restored coverage rather than to a suite change. Categories match the 2026-03-26 analyses so prior triage still applies. fail-fast: false so one language cannot hide the others. --- .github/workflows/codeql.yml | 113 +++++++++++++++++++++++++++++++++++ 1 file changed, 113 insertions(+) create mode 100644 .github/workflows/codeql.yml diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml new file mode 100644 index 000000000..a517d36c0 --- /dev/null +++ b/.github/workflows/codeql.yml @@ -0,0 +1,113 @@ +# CodeQL security analysis — advanced setup, in-repo and reviewable. +# +# No untrusted GitHub event inputs are used: the only expressions are +# `matrix.language` / `matrix.build-mode`, both defined in this file. +# +# WHY THIS FILE EXISTS +# +# CodeQL *security* analysis has not run on this repository since 2026-03-26. +# A workflow named "CodeQL" has been green every day since, which is why the +# gap was invisible. The green run is a different product: +# +# Initialize CodeQL: analysis-kinds: code-quality +# disable-default-queries: true +# - uses: code-quality +# +# That is GitHub's *code quality* default setup. It runs the maintainability +# suite over javascript-typescript and python and uploads `*.quality.sarif`. +# It produces no security results, and `code-scanning/default-setup` reports +# `state: not-configured` — the security half was switched off and only the +# quality half kept running. +# +# What the last real security scan (2026-03-26) covered, and found: +# +# /language:rust 178 results -> 100 alerts +# /language:actions 4 results +# /language:javascript-typescript 4 results +# /language:python 0 results +# /language:c-cpp 0 results +# +# All 100 rust alerts were triaged by the owner on 2026-03-26 and 2026-03-29 +# (50 "false positive", 50 "won't fix"). Dismissals are keyed by alert +# fingerprint and survive re-enablement, so restoring this workflow does not +# resurrect them — but it also means the current "0 open alerts" reflects that +# triage, not five months of clean scans. +# +# Everything merged since 2026-03-26 — the FORS address collision, the +# forged-signature verifier, the domain-separation cut, the Kyber binding +# enforcement, the BLE trust-root work — has never been seen by a security +# query. A settings toggle can be flipped with no diff and no notification; +# a committed workflow can only change through review. That is the fix. +name: CodeQL + +on: + push: + branches: [main] + pull_request: + branches: [main] + schedule: + # Weekly, so a language that stops analyzing surfaces as a failing + # scheduled run rather than as silence. + - cron: "27 4 * * 1" + +permissions: + contents: read + +jobs: + analyze: + name: CodeQL (${{ matrix.language }}) + runs-on: ubuntu-latest + # Rust and Kotlin are large here; give them room rather than letting a + # timeout masquerade as a language that "doesn't work". + timeout-minutes: 90 + permissions: + security-events: write + packages: read + actions: read + contents: read + + strategy: + # One language failing must not hide the others' results. + fail-fast: false + matrix: + include: + # build-mode: none analyses read source directly. That matters here: + # a full --all-features workspace build takes ~30 minutes, and + # requiring one is how Rust coverage gets quietly dropped again the + # next time the build is slow or breaks. + - language: rust + build-mode: none + # java-kotlin was never in the pre-2026-03-26 matrix, so the Android + # client (85 Kotlin files) has never been analyzed at all. + - language: java-kotlin + build-mode: none + - language: javascript-typescript + build-mode: none + - language: python + build-mode: none + - language: actions + build-mode: none + # c-cpp is deliberately NOT restored. It returned 0 results on + # 2026-03-26 because there is no first-party C to extract: the repo + # tracks 3 headers and no .c/.cpp. It also cannot run build-mode: + # none. Add it back with a real build mode if firmware C lands here. + + steps: + - name: Checkout + uses: actions/checkout@v7 + + - name: Initialize CodeQL + uses: github/codeql-action/init@v4 + with: + languages: ${{ matrix.language }} + build-mode: ${{ matrix.build-mode }} + # No `queries:` override — this is the same default suite that ran + # before 2026-03-26. Restoring coverage and changing the query suite + # in one step would make the resulting alert delta unattributable. + + - name: Analyze + uses: github/codeql-action/analyze@v4 + with: + # Same category strings as the 2026-03-26 analyses, so alerts land in + # the existing namespace and prior triage still applies. + category: "/language:${{ matrix.language }}" From c1918322305510f3cc6db8d6827e20c3c5272fae Mon Sep 17 00:00:00 2001 From: Cryptskii <47649969+cryptskii@users.noreply.github.com> Date: Fri, 14 Aug 2026 18:46:24 -0400 Subject: [PATCH 2/6] ci: build Kotlin for CodeQL, and get a full-tree run to prove coverage MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The first run of the restored workflow was not proof of anything. It came back rust=success/0 results, java-kotlin=FAILURE, and 0 results for every other language — where March recorded rust=178 and actions=4. Two distinct causes, one fixed here and one deliberately left alone. java-kotlin: build-mode: none cannot analyze Kotlin The extractor announced "Extracting Java with build-mode=none." — Java only. GitHub documents that buildless Java skips Kotlin because "Kotlin analysis requires a build". This repo has 0 .java files and 85 .kt, so buildless had nothing to extract: `database finalize` aborted with "CodeQL could not process any code written in Java/Kotlin." (exit code 32). The 2m29s that job spent was not a compile. The only Gradle tasks it ran were ForceDependencyResolutionPlugin_resolve* — the buildless classpath helper — which is why "BUILD SUCCESSFUL" appeared and no code was analyzed. The analysis row recorded at 22:19:46Z is not a real result either: it is the diagnostics-only ../codeql-failed-run.sarif uploaded by the post step after the failure (results=0, rules=0, error "unsuccessful execution"), versus rust's row at rules=25. Fixed with build-mode: manual plus a compile between init and analyze. Compile tasks rather than assembleDebug: refreshDsmJniLibs is hooked onto merge*JniLibFolders and hard-fails without cargo-ndk .so artifacts, which this job never builds. --no-daemon is load-bearing, not hygiene — a reused daemon starts outside the CodeQL tracer and its compilations go unobserved. JDK/Gradle setup mirrors ci.yml's android-unit-tests job, which builds this module green on ubuntu-latest today. results=0 everywhere: mostly diff scoping, not entirely codeql-action v4 logged "Persisted 1 diff range(s) across 1 file(s)" and passed --extension-packs=codeql-action/pr-diff-range to run-queries. PR #660 changes one YAML file, so source alerts are suppressed by construction. The same asymmetry exists in the March data on the same tooling: /language:rust returned 178 on refs/heads/main and 1 on refs/pull/85/head. But diff-informed analysis is opt-in per query, not global (observeDiffInformedIncrementalMode defaults to none()). rust/cleartext- logging opts in; rust/hard-coded-cryptographic-value, which produced 71 of March's 178, does not. Diff scoping alone does not explain its zero. The other candidate is extraction quality: 15,081 "macro expansion failed" warnings (assert_eq 3412, assert 2678, vec 2615, format 2300, log::* 1053) left 439 of 552 files carrying a warning and 113 clean. Macro bodies are absent from the AST — and that is precisely where the March alerts lived, in vec!/array literals and log!/format! arguments. So the rust job is left byte-identical on purpose. Changing it in the same commit that restores coverage would make the delta unattributable. The suspected one-line fix (rust-src for the missing sysroot) and the cfg-gated dsm_sdk/src/jni/ hole are both recorded as comments for a separate change. Proving it before merge workflow_dispatch only fires from the default branch, so it cannot produce a pre-merge full-tree run. This adds the branch to `push:` as an explicitly marked TEMPORARY PROBE, to be removed before merge, so the same workflow runs once without diff scoping and the rust count can be compared against March's 178 while this PR is still open. --- .github/workflows/codeql.yml | 85 +++++++++++++++++++++++++++++++++++- 1 file changed, 83 insertions(+), 2 deletions(-) diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index a517d36c0..ea1408b06 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -42,9 +42,20 @@ name: CodeQL on: push: - branches: [main] + # TEMPORARY PROBE — remove `ci/restore-codeql-security-analysis` before + # merge. A pull_request run cannot prove this workflow restores coverage: + # codeql-action v4 computes the PR diff ("Persisted 1 diff range(s) across + # 1 file(s)") and passes --extension-packs=codeql-action/pr-diff-range to + # run-queries, so a PR that changes only YAML reports 0 source alerts by + # construction. March shows the same asymmetry on the same tooling: + # /language:rust returned 178 on refs/heads/main and 1 on refs/pull/85/head. + # workflow_dispatch cannot substitute — it only fires from the default + # branch — so a push on this branch is the only pre-merge full-tree run. + branches: [main, ci/restore-codeql-security-analysis] pull_request: branches: [main] + # Lets a full-tree scan be forced without waiting for a push or the cron. + workflow_dispatch: schedule: # Weekly, so a language that stops analyzing surfaces as a failing # scheduled run rather than as silence. @@ -75,12 +86,39 @@ jobs: # a full --all-features workspace build takes ~30 minutes, and # requiring one is how Rust coverage gets quietly dropped again the # next time the build is slow or breaks. + # + # KNOWN DEGRADED, tracked separately — do not read a clean rust job + # as clean Rust coverage until this is closed. The first run logged + # 15,081 "macro expansion failed" warnings (assert_eq 3412, assert + # 2678, vec 2615, format 2300, log::* 1053), leaving 439 of 552 files + # carrying at least one warning and only 113 clean. Everything inside + # a macro invocation is absent from the AST, and that is exactly + # where the March alerts lived: hard-coded crypto in `vec!`/array + # literals, cleartext-logging sinks in `log::*`/`format!` arguments. + # Suspected cause is the missing `rust-src` component (the extractor + # logged `sysroot: None, sysroot_src: None, proc_macro_server: None`), + # which is a one-line experiment — but it is deliberately NOT applied + # in the same change as the coverage restoration, so the result delta + # stays attributable to one variable at a time. + # + # Separately, 31 files under dsm_sdk/src/jni/ and platform/{android, + # ios} got "semantic analyzer unavailable (not included as a module)" + # because they are cfg-gated to non-host targets. The JNI boundary is + # invisible to every security query until that is addressed. - language: rust build-mode: none # java-kotlin was never in the pre-2026-03-26 matrix, so the Android # client (85 Kotlin files) has never been analyzed at all. + # + # This one CANNOT use build-mode: none. Buildless Java extracts Java + # source only — the extractor announces "Extracting Java with + # build-mode=none." and GitHub documents that Kotlin is skipped in + # that mode because "Kotlin analysis requires a build". This repo has + # 0 .java files and 85 .kt, so buildless extracts nothing at all: the + # first attempt died in `database finalize` with "CodeQL could not + # process any code written in Java/Kotlin." (exit code 32). - language: java-kotlin - build-mode: none + build-mode: manual - language: javascript-typescript build-mode: none - language: python @@ -96,6 +134,27 @@ jobs: - name: Checkout uses: actions/checkout@v7 + # Mirrors the existing `android-unit-tests` job in ci.yml, which builds + # this module green on ubuntu-latest with the same JDK. + - name: Set up JDK 17 + if: matrix.language == 'java-kotlin' + uses: actions/setup-java@v5 + with: + distribution: temurin + java-version: "17" + cache: gradle + + - name: Cache Gradle caches + if: matrix.language == 'java-kotlin' + uses: actions/cache@v6 + with: + path: | + ~/.gradle/caches + ~/.gradle/wrapper + key: ${{ runner.os }}-gradle-${{ hashFiles('dsm_client/android/**/*.gradle*', 'dsm_client/android/**/gradle-wrapper.properties') }} + restore-keys: | + ${{ runner.os }}-gradle- + - name: Initialize CodeQL uses: github/codeql-action/init@v4 with: @@ -105,6 +164,28 @@ jobs: # before 2026-03-26. Restoring coverage and changing the query suite # in one step would make the resulting alert delta unattributable. + # Must run AFTER init and BEFORE analyze: manual mode only captures what + # the CodeQL tracer observes between those two steps. + # + # Compile tasks, not `assembleDebug`: app/build.gradle.kts hooks + # refreshDsmJniLibs onto merge*JniLibFolders, and that task hard-fails + # when the cargo-ndk .so artifacts are absent — which they always are + # here, since this job builds no Rust. Compile tasks never touch jniLibs. + # The three tasks together cover all 85 .kt files (main 58, test 20, + # androidTest 7). + # + # --no-daemon is required for correctness, not just hygiene: a reused + # Gradle daemon starts outside the CodeQL tracer, and compilations it + # performs would go unobserved. + - name: Compile Kotlin under the CodeQL tracer + if: matrix.language == 'java-kotlin' + working-directory: dsm_client/android + run: | + ./gradlew --no-daemon --stacktrace \ + :app:compileDebugKotlin \ + :app:compileDebugUnitTestKotlin \ + :app:compileDebugAndroidTestKotlin + - name: Analyze uses: github/codeql-action/analyze@v4 with: From b600e198017403b0376b20c4f15966a5ac388272 Mon Sep 17 00:00:00 2001 From: Cryptskii <47649969+cryptskii@users.noreply.github.com> Date: Fri, 14 Aug 2026 18:54:13 -0400 Subject: [PATCH 3/6] ci: restore Rust macro expansion, and scope the Kotlin build to what compiles MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The full-tree run proved the workflow is load-bearing and simultaneously proved it was under-reporting. On refs/heads/ci/restore-codeql-security- analysis, /language:rust returned 64 results against March's 178: rust/hard-coded-cryptographic-value 71 -> 64 rust/cleartext-logging 105 -> 0 rust/cleartext-transmission 2 -> 0 That is not a uniform shortfall, it is one query class deleted. The cleartext-logging sinks — log::__private_api::log, std::io::stdio::_print, core::panicking::panic_fmt — are reachable only through macro expansion, and the run logged 15,081 "macro expansion failed" warnings. Losing expansion removes the entire sink set, so the query cannot fire at all. This was the assay chosen before the run precisely because it discriminates "scan is degraded" from "scan is fine, PR diff scoping hid the results". Cause: the extractor logged `sysroot: None, sysroot_src: None` and rust-src is not preinstalled on the ubuntu runner image, so core/std are absent from the crate graph and every builtin macro fails. `rustup component add rust-src` targets the ~11,000 std-macro failures (assert_eq 3412, assert 2678, vec 2615, format 2300). The ~1,800 third-party macro_rules failures (log 1053, anyhow 448, rusqlite 321) need dependency sources and are left for a separate change so this one stays attributable. The 64 hard-coded-crypto results are the frozen pre-change baseline. If cleartext-logging returns on the next run, the sysroot hypothesis holds. Kotlin: the build now compiles, and androidTest is excluded build-mode: manual worked — :app:compileDebugKotlin and :app:compileDebugUnitTestKotlin both succeeded. The job still failed on :app:compileDebugAndroidTestKotlin with 14 "Unresolved reference" errors against protobuf accessors that no longer exist (reserveAU128, reserveBU128, setReserveAU128, setTokenId) in SoFiCrossDeviceOwnerTest.kt, SoFiTestHelpers.kt and SoFiTradeRealHwTest.kt. That breakage is pre-existing and invisible to CI: ci.yml runs :app:testDebugUnitTest, which compiles main and test but never androidTest, so nothing in the pipeline has ever compiled those 7 files. Tracked separately rather than fixed here. Blocking all Kotlin security coverage on 7 uncompilable instrumented tests would be the wrong trade — the remaining two tasks cover the 78 files that matter, all 58 production sources plus the 20 unit tests. --- .github/workflows/codeql.yml | 36 ++++++++++++++++++++++++++++++++++-- 1 file changed, 34 insertions(+), 2 deletions(-) diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index ea1408b06..d8ee92d92 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -134,6 +134,27 @@ jobs: - name: Checkout uses: actions/checkout@v7 + # Rust macro bodies are invisible to CodeQL without std/core sources. + # Measured on the 2026-08-14 full-tree run of this workflow WITHOUT this + # step: /language:rust returned 64 results, all rust/hard-coded- + # cryptographic-value — while rust/cleartext-logging returned 0 against + # 105 in March, and rust/cleartext-transmission 0 against 2. That is the + # expected signature of 15,081 "macro expansion failed" warnings: the + # cleartext-logging sinks (log::__private_api::log, std::io::stdio:: + # _print, core::panicking::panic_fmt) are reachable ONLY through macro + # expansion, so losing expansion deletes the entire query's sink set. + # The extractor logged `sysroot: None, sysroot_src: None`, and rust-src + # is not preinstalled on the ubuntu runner image. + # + # This targets the ~11,000 std-macro failures (assert_eq 3412, assert + # 2678, vec 2615, format 2300). The ~1,800 third-party macro_rules + # failures (log 1053, anyhow 448, rusqlite 321) need dependency sources + # and are a separate variable — deliberately not changed in the same + # step, so the effect of this one stays attributable. + - name: Provide Rust sysroot sources for the CodeQL extractor + if: matrix.language == 'rust' + run: rustup component add rust-src + # Mirrors the existing `android-unit-tests` job in ci.yml, which builds # this module green on ubuntu-latest with the same JDK. - name: Set up JDK 17 @@ -177,14 +198,25 @@ jobs: # --no-daemon is required for correctness, not just hygiene: a reused # Gradle daemon starts outside the CodeQL tracer, and compilations it # performs would go unobserved. + # + # androidTest is deliberately EXCLUDED. Those 7 instrumented-test files + # do not compile today — :app:compileDebugAndroidTestKotlin fails with 14 + # "Unresolved reference" errors against protobuf accessors that no longer + # exist (reserveAU128, reserveBU128, setReserveAU128, setTokenId) in + # SoFiCrossDeviceOwnerTest.kt, SoFiTestHelpers.kt and SoFiTradeRealHwTest + # .kt. Nothing in CI compiles them: ci.yml runs :app:testDebugUnitTest, + # which covers main and test but never androidTest. That breakage is + # pre-existing and tracked separately — it is not this workflow's to fix, + # and blocking all Kotlin security coverage on it would be the wrong + # trade. The two tasks below cover the 78 files that matter: all 58 + # production sources plus the 20 unit tests. - name: Compile Kotlin under the CodeQL tracer if: matrix.language == 'java-kotlin' working-directory: dsm_client/android run: | ./gradlew --no-daemon --stacktrace \ :app:compileDebugKotlin \ - :app:compileDebugUnitTestKotlin \ - :app:compileDebugAndroidTestKotlin + :app:compileDebugUnitTestKotlin - name: Analyze uses: github/codeql-action/analyze@v4 From f26bfb5f97b215d2ee0ad851dd8bbe3f97d2c769 Mon Sep 17 00:00:00 2001 From: Cryptskii <47649969+cryptskii@users.noreply.github.com> Date: Fri, 14 Aug 2026 19:38:39 -0400 Subject: [PATCH 4/6] ci: drop the rust-src step, which measurement refuted MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Installing rust-src changed nothing. With the component present the run was byte-identical to the run without it: 15,081 "macro expansion failed" warnings, 439 files with warnings / 113 clean, and the same 64 results (all rust/hard-coded-cryptographic-value, still no cleartext-logging). The extractor continued to report `sysroot: None, sysroot_src: None, rustc_src: None, proc_macro_server: None` — those are unset config OPTIONS, not the result of failed discovery, so reading them as "rust-src is missing" was the wrong inference. Removing it rather than keeping it: a step with no measured effect is dead config, and the comment justifying it asserted something the measurement disproved. The finding is preserved where it belongs — on the rust matrix entry, with the before/after numbers and the refuted hypothesis recorded so it is not retried blindly. Next candidates, one variable at a time against the frozen baseline of 64: explicit CODEQL_EXTRACTOR_RUST_OPTION_* overrides, and fetched dependency sources for the third-party macro_rules failures. --- .github/workflows/codeql.yml | 58 ++++++++++++++++-------------------- 1 file changed, 25 insertions(+), 33 deletions(-) diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index d8ee92d92..79a5d625a 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -88,18 +88,31 @@ jobs: # next time the build is slow or breaks. # # KNOWN DEGRADED, tracked separately — do not read a clean rust job - # as clean Rust coverage until this is closed. The first run logged - # 15,081 "macro expansion failed" warnings (assert_eq 3412, assert - # 2678, vec 2615, format 2300, log::* 1053), leaving 439 of 552 files - # carrying at least one warning and only 113 clean. Everything inside - # a macro invocation is absent from the AST, and that is exactly - # where the March alerts lived: hard-coded crypto in `vec!`/array - # literals, cleartext-logging sinks in `log::*`/`format!` arguments. - # Suspected cause is the missing `rust-src` component (the extractor - # logged `sysroot: None, sysroot_src: None, proc_macro_server: None`), - # which is a one-line experiment — but it is deliberately NOT applied - # in the same change as the coverage restoration, so the result delta - # stays attributable to one variable at a time. + # as clean Rust coverage until this is closed. Measured on the + # 2026-08-14 full-tree run: 15,081 "macro expansion failed" warnings + # (assert_eq 3412, assert 2678, vec 2615, format 2300, log::* 1053), + # leaving 439 of 552 files carrying at least one warning and 113 + # clean. Everything inside a macro invocation is absent from the AST, + # and that is exactly where the March findings lived: + # + # rust/hard-coded-cryptographic-value 71 -> 64 (mostly intact) + # rust/cleartext-logging 105 -> 0 (sink set gone) + # rust/cleartext-transmission 2 -> 0 + # + # cleartext-logging's sinks — log::__private_api::log, + # std::io::stdio::_print, core::panicking::panic_fmt — exist only + # inside macro expansions, so losing expansion deletes the query + # outright rather than shrinking it. + # + # `rustup component add rust-src` was TRIED AND MEASURED, and is not + # the fix: with the component installed, the run was byte-identical + # (15,081 failures, 439/113, same 64 results). The extractor still + # reported `sysroot: None, sysroot_src: None, rustc_src: None, + # proc_macro_server: None, extract_dependencies_as_source: false` — + # those are unset options, not failed discovery. Next candidates are + # the explicit CODEQL_EXTRACTOR_RUST_OPTION_* overrides and fetched + # dependency sources, one variable at a time against the frozen + # baseline of 64. # # Separately, 31 files under dsm_sdk/src/jni/ and platform/{android, # ios} got "semantic analyzer unavailable (not included as a module)" @@ -134,27 +147,6 @@ jobs: - name: Checkout uses: actions/checkout@v7 - # Rust macro bodies are invisible to CodeQL without std/core sources. - # Measured on the 2026-08-14 full-tree run of this workflow WITHOUT this - # step: /language:rust returned 64 results, all rust/hard-coded- - # cryptographic-value — while rust/cleartext-logging returned 0 against - # 105 in March, and rust/cleartext-transmission 0 against 2. That is the - # expected signature of 15,081 "macro expansion failed" warnings: the - # cleartext-logging sinks (log::__private_api::log, std::io::stdio:: - # _print, core::panicking::panic_fmt) are reachable ONLY through macro - # expansion, so losing expansion deletes the entire query's sink set. - # The extractor logged `sysroot: None, sysroot_src: None`, and rust-src - # is not preinstalled on the ubuntu runner image. - # - # This targets the ~11,000 std-macro failures (assert_eq 3412, assert - # 2678, vec 2615, format 2300). The ~1,800 third-party macro_rules - # failures (log 1053, anyhow 448, rusqlite 321) need dependency sources - # and are a separate variable — deliberately not changed in the same - # step, so the effect of this one stays attributable. - - name: Provide Rust sysroot sources for the CodeQL extractor - if: matrix.language == 'rust' - run: rustup component add rust-src - # Mirrors the existing `android-unit-tests` job in ci.yml, which builds # this module green on ubuntu-latest with the same JDK. - name: Set up JDK 17 From 32a1f45a7c30dadc402eeb5600b704f3a79db6fa Mon Sep 17 00:00:00 2001 From: Cryptskii <47649969+cryptskii@users.noreply.github.com> Date: Fri, 14 Aug 2026 20:04:09 -0400 Subject: [PATCH 5/6] ci: pin the extractor's Rust sysroot to 1.94 so std macros expand MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Root cause of the missing 107 Rust findings is upstream, and it is neither build-mode nor a missing component. The CodeQL bundle's vendored rust-analyzer cannot expand std macros from Rust >= 1.95.0. Bundle 2.26.3 pins ra_ap_* 0.0.301; the upgrade to 0.328 that fixes it (github/codeql #21714) merged upstream on 2026-08-14 and is in no released bundle yet. rust-toolchain.toml pins channel = "1.96.0". The extractor resolves its sysroot by running `rustc --print sysroot` in the checkout root, where rustup honours that file, so it lands on exactly the broken version. The failure signature matches the upstream reproduction exactly. github/ codeql#19982 varied only rust-toolchain.toml on a five-line crate: 1.85, 1.92 and 1.94 produced zero failures; 1.96.0 produced vec, assert, format and $crate::format_args_nl; 1.97.0 added cfg_select and pattern_type. This tree logs vec 2615, assert 2678, assert_eq 3412, format 2300, $crate::format_args_nl 377 — and zero cfg_select, zero pattern_type. That is the 1.96 row, not 1.97, and this repo pins 1.96.0. The third-party failures are downstream of the same break: log::warn 529, log::info 400, anyhow 448 and rusqlite::params 321 are macro_rules that expand to format_args!/vec! internally, so they fail transitively once the std definitions do. Together with the std classes that is ~13,563 of the 15,081 warnings. Both variables are required. SYSROOT_SRC alone takes the (None, Some) arm of the extractor's sysroot match and keeps the discovered 1.96 binary sysroot; SYSROOT alone takes discover_rust_lib_src_dir, which unlike the (None, None) arm does not auto-install rust-src and then errors when the sources are absent. This redirects only the extractor. No other job's toolchain changes and rust-toolchain.toml is untouched. Also records the two refuted hypotheses on the matrix entry so they are not retried: rust-src (the extractor installs it itself, which is why that run was byte-identical) and a build-mode change (the extractor never reads build output). And it corrects the reading that misled the first attempt — `sysroot: None` means the option is unset and discovery ran, not that discovery failed. Expected against the frozen baseline of 64 results / 15,081 failures: warnings collapse to the low hundreds, files-with-warnings fall from 439/113 toward roughly 15/537, cleartext-logging returns near 105, cleartext-transmission near 2, hard-coded-cryptographic-value 64 -> ~71. Revert once a bundle ships ra_ap >= 0.328, and re-measure rather than assume the numbers hold. --- .github/workflows/codeql.yml | 46 +++++++++++++++++++++++++++++------- 1 file changed, 37 insertions(+), 9 deletions(-) diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index 79a5d625a..cf1753d8b 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -104,15 +104,18 @@ jobs: # inside macro expansions, so losing expansion deletes the query # outright rather than shrinking it. # - # `rustup component add rust-src` was TRIED AND MEASURED, and is not - # the fix: with the component installed, the run was byte-identical - # (15,081 failures, 439/113, same 64 results). The extractor still - # reported `sysroot: None, sysroot_src: None, rustc_src: None, - # proc_macro_server: None, extract_dependencies_as_source: false` — - # those are unset options, not failed discovery. Next candidates are - # the explicit CODEQL_EXTRACTOR_RUST_OPTION_* overrides and fetched - # dependency sources, one variable at a time against the frozen - # baseline of 64. + # Cause: the bundle's rust-analyzer cannot expand std macros from + # Rust >= 1.95.0, and rust-toolchain.toml pins 1.96.0. Addressed by + # the sysroot pin in the steps below; see that comment for detail. + # + # Two hypotheses were tried and refuted, recorded so they are not + # retried: `rustup component add rust-src` (byte-identical run — + # the extractor already installs that component itself), and a + # `build-mode` change (the extractor never reads build output, it + # re-derives everything through rust-analyzer and hits the same + # sysroot). `sysroot: None` in the extractor's config dump means the + # option is unset and discovery ran, NOT that discovery failed — + # misreading that line is what sent the first attempt wrong. # # Separately, 31 files under dsm_sdk/src/jni/ and platform/{android, # ios} got "semantic analyzer unavailable (not included as a module)" @@ -147,6 +150,31 @@ jobs: - name: Checkout uses: actions/checkout@v7 + # The CodeQL bundle's vendored rust-analyzer cannot expand std macros + # from Rust >= 1.95.0. Bundle 2.26.3 pins ra_ap_* 0.0.301; the upgrade to + # 0.328 that fixes it (github/codeql#21714) merged upstream on + # 2026-08-14 and is in no released bundle yet. + # + # rust-toolchain.toml pins channel = "1.96.0", and the extractor resolves + # its sysroot by running `rustc --print sysroot` in the checkout root — + # where rustup honours that file — so it lands on exactly the broken + # version. Both variables are required: SYSROOT_SRC alone keeps the + # discovered 1.96 binary sysroot, and SYSROOT alone skips the automatic + # rust-src install and then errors when the sources are absent. + # + # This redirects ONLY the extractor. It does not change the toolchain any + # other job builds with, and rust-toolchain.toml is untouched. + # + # Revert this once a bundle ships ra_ap >= 0.328, and confirm the numbers + # hold rather than assuming they do. + - name: Pin the Rust sysroot read by the CodeQL extractor + if: matrix.language == 'rust' + run: | + rustup toolchain install 1.94.0 --profile minimal --component rust-src + SYSROOT="$(rustup run 1.94.0 rustc --print sysroot)" + echo "CODEQL_EXTRACTOR_RUST_OPTION_SYSROOT=$SYSROOT" >> "$GITHUB_ENV" + echo "CODEQL_EXTRACTOR_RUST_OPTION_SYSROOT_SRC=$SYSROOT/lib/rustlib/src/rust/library" >> "$GITHUB_ENV" + # Mirrors the existing `android-unit-tests` job in ci.yml, which builds # this module green on ubuntu-latest with the same JDK. - name: Set up JDK 17 From 4b77de6fb78d03e003817046a7a4ec114914a6aa Mon Sep 17 00:00:00 2001 From: Cryptskii <47649969+cryptskii@users.noreply.github.com> Date: Fri, 14 Aug 2026 20:15:54 -0400 Subject: [PATCH 6/6] ci: remove the temporary probe trigger, and record the measured outcome MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The sysroot pin worked. Full-tree runs on this branch, before and after: macro fails files w/ warnings results before sysroot pin 15,081 439 / 113 64 after sysroot pin 18 48 / 504 259 rule March before after rust/cleartext-logging 105 0 145 rust/hard-coded-cryptographic-value 71 64 112 rust/cleartext-transmission 2 0 2 That is 99.9% of the macro failures gone and the result count now above the March baseline of 178 — expected, since the tree has five months of new code and the rust-queries pack moved 0.1.28 -> 0.1.40. All five jobs green. The 18 residual failures are all in the no_std embedded crates (hal::binary_info::* in dsm-anchor-pico), which need their thumb target to resolve. Recorded on the matrix entry alongside the still-open cfg-gated JNI gap, rather than left implicit in a green check. The probe trigger has served its purpose and is removed, so `push:` is back to main only. workflow_dispatch stays — it is the supported way to force a full-tree scan without waiting for a push or the weekly cron, and it now carries a note that a green PR check is not evidence of coverage, because codeql-action restricts PR alerts to the diff. --- .github/workflows/codeql.yml | 56 ++++++++++++++++++------------------ 1 file changed, 28 insertions(+), 28 deletions(-) diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index cf1753d8b..9cb4dc478 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -42,19 +42,14 @@ name: CodeQL on: push: - # TEMPORARY PROBE — remove `ci/restore-codeql-security-analysis` before - # merge. A pull_request run cannot prove this workflow restores coverage: - # codeql-action v4 computes the PR diff ("Persisted 1 diff range(s) across - # 1 file(s)") and passes --extension-packs=codeql-action/pr-diff-range to - # run-queries, so a PR that changes only YAML reports 0 source alerts by - # construction. March shows the same asymmetry on the same tooling: - # /language:rust returned 178 on refs/heads/main and 1 on refs/pull/85/head. - # workflow_dispatch cannot substitute — it only fires from the default - # branch — so a push on this branch is the only pre-merge full-tree run. - branches: [main, ci/restore-codeql-security-analysis] + branches: [main] pull_request: branches: [main] # Lets a full-tree scan be forced without waiting for a push or the cron. + # Worth knowing when reading a PR check: codeql-action restricts alerts to + # the PR diff ("Persisted N diff range(s)"), so a PR that touches no Rust + # reports no Rust alerts by construction. A green PR check is not evidence + # of coverage — the push, dispatch and cron runs are the full-tree ones. workflow_dispatch: schedule: # Weekly, so a language that stops analyzing surfaces as a failing @@ -87,36 +82,41 @@ jobs: # requiring one is how Rust coverage gets quietly dropped again the # next time the build is slow or breaks. # - # KNOWN DEGRADED, tracked separately — do not read a clean rust job - # as clean Rust coverage until this is closed. Measured on the - # 2026-08-14 full-tree run: 15,081 "macro expansion failed" warnings - # (assert_eq 3412, assert 2678, vec 2615, format 2300, log::* 1053), - # leaving 439 of 552 files carrying at least one warning and 113 - # clean. Everything inside a macro invocation is absent from the AST, - # and that is exactly where the March findings lived: + # Rust coverage depends on macro expansion working, and it silently + # did not. Measured progression on this branch, full-tree runs: + # + # macro fails files w/ warnings results + # before sysroot pin 15,081 439 / 113 64 + # after sysroot pin 18 48 / 504 259 # - # rust/hard-coded-cryptographic-value 71 -> 64 (mostly intact) - # rust/cleartext-logging 105 -> 0 (sink set gone) - # rust/cleartext-transmission 2 -> 0 + # rule March before after + # rust/cleartext-logging 105 0 145 + # rust/hard-coded-cryptographic-value 71 64 112 + # rust/cleartext-transmission 2 0 2 # # cleartext-logging's sinks — log::__private_api::log, # std::io::stdio::_print, core::panicking::panic_fmt — exist only - # inside macro expansions, so losing expansion deletes the query - # outright rather than shrinking it. - # - # Cause: the bundle's rust-analyzer cannot expand std macros from - # Rust >= 1.95.0, and rust-toolchain.toml pins 1.96.0. Addressed by - # the sysroot pin in the steps below; see that comment for detail. + # inside macro expansions, so a broken expander deleted that query + # outright instead of shrinking it. A rule returning 0 is the signal + # to check extraction health, not evidence of clean code. # # Two hypotheses were tried and refuted, recorded so they are not - # retried: `rustup component add rust-src` (byte-identical run — - # the extractor already installs that component itself), and a + # retried: `rustup component add rust-src` (byte-identical run — the + # extractor already installs that component itself), and a # `build-mode` change (the extractor never reads build output, it # re-derives everything through rust-analyzer and hits the same # sysroot). `sysroot: None` in the extractor's config dump means the # option is unset and discovery ran, NOT that discovery failed — # misreading that line is what sent the first attempt wrong. # + # Residual, tracked separately: 18 macro failures remain, all in the + # no_std embedded crates (hal::binary_info::* in dsm-anchor-pico), + # which need their thumb target to resolve. Separately, ~31 files + # under dsm_sdk/src/jni/ and platform/{android,ios} are cfg-gated to + # non-host targets and get no semantic analysis at all, so the JNI + # boundary is still invisible; the lever there is CARGO_CFG_OVERRIDES + # or a second matrix entry with a distinct category. + # # Separately, 31 files under dsm_sdk/src/jni/ and platform/{android, # ios} got "semantic analyzer unavailable (not included as a module)" # because they are cfg-gated to non-host targets. The JNI boundary is