Skip to content

fix(gate): stop the untested-data-access gate false-positiving on non-SQL text - #214

Merged
veksen merged 2 commits into
mainfrom
fix-select-shape-false-positives
Aug 3, 2026
Merged

fix(gate): stop the untested-data-access gate false-positiving on non-SQL text#214
veksen merged 2 commits into
mainfrom
fix-select-shape-false-positives

Conversation

@veksen

@veksen veksen commented Aug 3, 2026

Copy link
Copy Markdown
Member

Goal

The untested-data-access gate should only fail a build when a PR changes a query that no real-DB test exercises. It is fail by default (packages/core/src/ci/policy.ts:50 in Site), so a false positive blocks a merge that has no data-layer test to add. This PR fixes four false positives reported from dogfooding on our own repo: Query-Doctor/Site#3615 and Query-Doctor/Site#3650.

The precision umbrella is Query-Doctor/Site#3503 (capture-based detection), which replaces this diff heuristic outright. Until that lands, these misfires block merges today. This PR is interim work on the heuristic, not a substitute for #3503.

What

Before: a frontend-only PR that touched no database could fail the build.

  • Query-Doctor/Site#3811 (a mobile CSS pass) was flagged on apps/app/src/routes/_app/$username/$projectSlug/queries/index.tsx, a React route component with no SQL, ORM or db import.
  • Query-Doctor/Site#3649 (our staging-to-prod merge) was flagged on two .tsx components and on two apps/api/scripts/*.sql files that each ship a real-DB spec in the same PR.

After: neither PR produces a verdict. Genuine data-access changes are still flagged exactly as before.

How

Read src/gate/test-presence.ts first. The two commits are separate mechanisms and are worth reviewing in order.

Commit 1 stops non-SQL text reaching the matcher. Every .tsx flag came from one pattern, the raw select … from shape, matching ordinary TypeScript:

  • import { Button, Select } from "@query-doctor/ui" puts Select } from in the text. This is a UI component named Select, not the react-query *.queries.ts import that Query-Doctor/Site#3650 suspected.
  • export { default as Select } from "./components/select" in a barrel file hits the same shape one keyword over.
  • A JSX comment reading "…and select stay one unwrappable unit… must not wrap away from the control it labels" hits it too. Line-based comment stripping missed it twice: the opening line trims to {/*, not /*, and the continuation lines are bare prose with no * decoration. The quer(y|ies) content match hypothesised in Query-Doctor/Site#3615 does not exist in the config and was not involved.

The fix strips /* … */ spans and module imports before matching. The re-export rule keys on export { and export * rather than a bare export prefix, so a declaration like export const q = sql`SELECT id FROM "users"` is still inspected. That boundary is the one real recall risk in this PR and has its own test.

Commit 2 credits a spec that lives in another directory. baseStem stripped .ts, .py, .go and .rb but not .sql, so it compared backfill-personal-teams.sql against the spec's backfill-personal-teams and found no match. That spec reads the script file and runs it against Postgres. Adding .sql widens an already lenient relation, which is the direction the rule documents: a loose match makes the gate under-fire, the safe side.

What I did not fix

Sweeping every non-test file in Site's apps/app/src, apps/blog/src and packages/ui/src as if fully added, 7 of 415 still classify as data access. All 7 hold SQL as content rather than as a query they run: query fixtures, the rendered CREATE INDEX CONCURRENTLY ON recommendation string, planner-node descriptions, demo SQL in a blog component.

No regex separates "SQL text present" from "SQL text executed". The blunt fix Query-Doctor/Site#3650 proposes, excluding frontend .tsx from the declared set, would create genuine false negatives, because Next.js server components query databases from .tsx. This class is evidence for #3503 rather than something to patch here.

Tests

Five tests in src/gate/test-presence.test.ts, each written before its fix and observed failing first. Four pin a reported false positive. The fifth pins the recall boundary of the re-export rule, and was mutation-checked: it fails under a bare export-prefix rule and passes as written.

Full suite 410/410. npm run typecheck clean.

Beyond the unit tests, I replayed both PRs' real changed-file lists from the GitHub API through evaluateTestPresence. On current main the gate fires on 1 file for Query-Doctor/Site#3811 and 4 for Query-Doctor/Site#3649, matching both bug reports exactly. With this branch, both return no verdict.

For recall, I replayed the last 40 merged Site PRs plus those two. Classification drops exactly the three frontend .tsx files and keeps all 13 genuine data-access files, so nothing real stopped being detected.

Refs Query-Doctor/Site#3615, Query-Doctor/Site#3650

veksen and others added 2 commits August 3, 2026 12:50
The untested-data-access gate flagged two frontend-only PRs and failed the
build on changes that cannot have a data-layer test. One pattern caused every
flag: the raw `select … from` shape matched ordinary TypeScript, not SQL.

`import { Button, Select } from "@query-doctor/ui"` puts `Select } from` in the
text (Site#3650). That is a UI component named Select, not the react-query
import the issue suspected. `export { default as Select } from "./select"` in a
barrel file hits the same shape one keyword over.

A JSX comment about a sort control puts "select stay … away from" there
(Site#3615). Line-based comment stripping missed it twice. The opening line
trims to `{/*`, not `/*`, and the continuation lines are bare prose with no `*`
decoration.

Strip `/* … */` spans and module imports before matching. The re-export rule
matches `export {` and `export *` rather than a bare `export` prefix, so a
declaration like ``export const q = sql`SELECT id FROM "users"` `` is still
inspected; a test pins that boundary.

Replaying both PRs' real changed-file lists through the gate now returns no
verdict, where the current main fires on one file for the first and four for
the second. Across 40 merged Site PRs plus those two, classification drops
exactly the three frontend files and keeps all 13 genuine data-access files.

Refs Query-Doctor/Site#3615, Query-Doctor/Site#3650

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An operational script and the spec that runs it sit in different directories,
so the stem rule is the only one that can link them. `baseStem` stripped
`.ts`, `.py`, `.go` and `.rb` but not `.sql`, so it compared
`backfill-personal-teams.sql` against the spec's `backfill-personal-teams` and
found no match. The gate then reported the script as untested and failed the
build (Site#3650).

Strip `.sql` too. Replaying the reported PR's real changed-file list now
returns no verdict, where the current main flags both scripts despite each
one shipping a spec that reads the file and runs it against Postgres.

This widens an already lenient relation, which is the direction the rule
documents: a loose match makes the gate under-fire, the safe side.

Refs Query-Doctor/Site#3650

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Query Doctor — 6 successful checks

 Cost regression — No query went up more than 5%
 Untested data access — No changed data-access file without a test
 New query — No new queries
 New query with index recommendation — No new query ships an index recommendation
 Schema drift — No schema changes
 High-value nudge — No index or rewrite past the threshold


More details via MCP → get_ci_run({ runId: "019fc859-3c55-70bc-9a01-d7c8414d8caf" }) · view run · docs
3 queries read against main on assumed statistics of 10,000,000 rows per table. Sync production stats for costs measured against your real data.

@veksen
veksen merged commit 081767c into main Aug 3, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant