Conversation
|
/bot-deep-review |
1 similar comment
|
/bot-deep-review |
|
@davidda could you please rebase this on latest master? Thanks |
f998c76 to
3f6e271
Compare
|
@MazterQyou rebase done |
|
/bot-deep-review |
|
Claude finished @MazterQyou's task in 12m 6s —— View job Deep review — approach is sound, no correctness bug found. 0 high, 3 medium, 3 low (6 inline comments).Full reviewTodo
VerdictThe design holds up. I traced every context the flag reaches, looking specifically for a slot where Things I checked that turned out fine, worth recording so nobody re-derives them:
Security: the templates are static and Findings
The first one is the one I'd act on. The truth-table test round-trips the MSSQL SQL through DataFusion after a Not verifiedI could not compile or run anything here: |
3f6e271 to
045c34f
Compare
|
/bot-deep-review |
|
Claude finished @MazterQyou's task in 14m 38s —— View job Deep review — approach is sound; 1 medium, 3 low, plus a red
|
| # | Severity | Where |
|---|---|---|
| 1 | medium | rust/cubesql/cubesql/src/compile/test/mod.rs:1501 — test fixture include_str!-ed into release builds from outside the Rust workspace |
| 2 | low | packages/cubejs-schema-compiler/src/adapter/MssqlQuery.ts:328 — predicate_to_scalar doubles the SQL per nesting level, unbounded |
| 3 | low | packages/cubejs-schema-compiler/test/integration/mssql/mssql-boolean-contexts.test.ts:13 — ~58 full fixture rebuilds for tables the suite never reads |
| 4 | low | rust/cubesql/cubesql/src/compile/engine/df/wrapper/boolean_context_tests.rs:207 — permanently #[ignore]d test is the only record of the remaining IS TRUE gap |
CI: Lerna tsc is failing on this head
Run 34984929543 (commit 045c34f) fails the Lerna tsc step in five jobs — unit (24.x, 3.13), Build & Test :dev for Debian, integration-cubestore, and both integration-smoke matrix legs. lint, build-cubestore and Build native linux pass, so it is a TypeScript compile error, not a lint or Rust failure.
I could not read the job log in this environment (log download and gh run view --log-failed are both blocked here), so I can't attribute it. Worth ruling out the two new .json-importing test files first — they are the package's only JSON imports, and booleanFixture.cases / booleanFixture.aggregates get heterogeneous inferred element types from resolveJsonModule, which is where union-of-array-types method calls like test.expected.filter(...) (mssql-boolean-contexts.test.ts:25) tend to break. If master is red for an unrelated reason, ignore this.
Not verified
cargo and node_modules are both unavailable here, so I did not compile the Rust crate, run cargo test -p cubesql, or run the Jest suites. Every claim above is from reading the code; the MSSQL integration test in particular is unexecuted on my side.
Prior threads
All 7 existing review threads are already resolved; nothing to resolve and no finding duplicates one.
|
/bot-deep-review |
|
Claude finished @MazterQyou's task in 15m 46s —— View job Deep review complete — 7 findings posted inline: 1 high, 3 medium, 3 low. Core rendering design is sound; the linked issue is only partly fixed.Full review
What holds upThe context-threading design is contained and correct where it applies. Specific things I checked and found sound:
Findings
The high finding is the one that decides whether this PR closes #11826. The issue's model is a Re-affirmed, not re-posted
Not verifiedI did not execute anything. No |
|
/bot-deep-review |
|
Claude finished @MazterQyou's task in 15m 51s —— View job Deep review — 1 medium (correctness, non-MSSQL dialects), 1 docs gap, 2 comment-hygiene. 4 inline comments posted. Review details
Findings
What holds upThe core design is sound. Three-valued semantics survive both conversions — I checked one thing the duplication in Blast radius outside MSSQL is small: every new behaviour keys off Raised before, not re-posted
I also dropped a candidate finding about Not verifiedNeither the Rust nor the JS suite was run: no Review threadsListing returned 25 threads authored by |
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
|
/bot-deep-review |
|
Claude finished @MazterQyou's task in 14m 54s —— View job Deep review — no blocking issues. 0 high, 0 medium, 4 low (2 efficiency, 2 test/CI hygiene), posted inline. Full review
VerdictThe design holds up. I traced every one of the 48 Cross-dialect safety is real, not assumed: Two things I checked specifically because they would have been silent-wrong-answer bugs:
The documented workaround is backed by a planner-level test rather than just prose, which matters more than it looks: DataFusion's simplifier folds
Findings (all low)
Nothing here blocks merge. Not re-raisedAll 29 prior review threads on this PR are resolved, so nothing was skipped as a duplicate and nothing needed resolving. Two points remain open by the author's explicit, reasoned deferral and I did not relitigate them: the size growth of nested Not verifiedNo build or test run. This checkout has no |
| .is_some_and(|relation| context.known_join_subqueries.contains(relation)) | ||
| { | ||
| if let MemberField::Member(member) = | ||
| Self::find_member_in_ungrouped_scan(context.ungrouped_scan_node, column)? |
There was a problem hiding this comment.
find_member_in_ungrouped_scan now runs twice for every column on the MSSQL push-to-cube path: once here, and again inside generate_sql_for_column (rust/cubesql/cubesql/src/compile/engine/df/wrapper.rs:3084) after generate_sql_for_expr_raw dispatches. It is a linear zip-scan over ungrouped_scan_node.schema.fields(), so a wide ungrouped scan projecting most of its members goes from O(n²) to 2·O(n²) field comparisons, and each column additionally pays find_cube_with_name (linear over cubes) plus lookup_segment and lookup_dimension (linear, eq_ignore_ascii_case per member).
The guard conditions here are character-for-character the ones generate_sql_for_column already applies (subqueries miss, then known_join_subqueries miss, then the lookup), so the resolved MemberField could be threaded through instead of resolved twice — e.g. resolve once here and pass the result into generate_sql_for_expr_raw/generate_sql_for_column, or cache it on the context.
Non-blocking, and MSSQL-only, but it is pure duplicated work on the hottest per-expression path.
| #[tokio::test] | ||
| #[ignore = "DataFusion truth-test lowering loses NULL semantics before SQL rendering"] | ||
| async fn boolean_context_truth_tests_nulls() { | ||
| let ctx = SessionContext::new(); | ||
| let mut actual = vec![]; | ||
| for test in ["IS TRUE", "IS FALSE", "IS NOT TRUE", "IS NOT FALSE"] { | ||
| actual.extend(values(&ctx, &format!("SELECT CAST(NULL AS BOOLEAN) {test}")).await); | ||
| } | ||
| assert_eq!( | ||
| actual, | ||
| vec![Some(false), Some(false), Some(true), Some(true)] | ||
| ); | ||
| } |
There was a problem hiding this comment.
boolean_context_truth_tests_nulls never touches the code this PR changes. It builds a bare SessionContext, runs SELECT CAST(NULL AS BOOLEAN) IS TRUE and friends, and asserts DataFusion's own results — no render(...), no generate_sql_for_expr_context, no MSSQL templates. Combined with #[ignore], it is inert in CI and, if someone ever un-ignores it after a DataFusion bump, a pass tells them the lowering bug is gone but nothing about whether IS [NOT] TRUE/FALSE renders correctly for MSSQL — which is the thing this file exists to pin.
Making it a real reproducer for this PR's boundary is a small change: plan the expression through ctx.sql, then assert on render(expr, true, true) / render(expr, false, true) alongside the DataFusion values, the way boolean_context_truth_tables does. Then un-ignoring it after an upgrade actually exercises the renderer.
| "expected": 1 | ||
| } | ||
| ], | ||
| "dimensionCases": [ |
There was a problem hiding this comment.
dimensionCases carries the model/expected axis that only the MSSQL integration test reads, but boolean_context_segment_members (rust/cubesql/cubesql/src/compile/test/test_wrapper.rs) iterates the same array and asserts sql alone. Three pairs are byte-identical in both query and sql, so Rust plans the same query twice and asserts the same expectation twice:
WHERE has_subscription GROUP BY 1→(${…has_subscription})at lines 333 and 431SUM(CASE WHEN has_subscription …)→SUM(CASE WHEN (${…has_subscription}) …)at lines 347 and 354COUNT(DISTINCT has_subscription)→COUNT(DISTINCT ${…has_subscription})at lines 417 and 439
That is 3 redundant full convert_sql_to_cube_query runs out of 15 in a test that already re-plans every case. Deduplicating on the Rust side — e.g. collect the distinct (query, sql) pairs before the loop — keeps the JS matrix intact while dropping a fifth of the planner work.
| - 'packages/cubejs-schema-compiler/src/adapter/MssqlQuery.ts' | ||
| - 'packages/cubejs-schema-compiler/test/fixtures/mssql-boolean-contexts.json' |
There was a problem hiding this comment.
No Rust code reads MssqlQuery.ts. The only cross-tree dependency is the fixture, pulled in by include_str! at rust/cubesql/cubesql/src/compile/test/mod.rs:1502 — so line 9 is load-bearing and line 8 is not. Drift between the adapter and the fixture is caught on the JS side by packages/cubejs-schema-compiler/test/unit/mssql-query.test.ts:8, which compares MssqlQuery.prototype.sqlTemplates() against booleanFixture.templates.
Net effect of line 8 (and its pull_request twin at line 18): every edit to MssqlQuery.ts — null ordering, DATE_ADD, PERCENTILECONT, anything — now triggers a full native build plus the cubesql suite, for a file the suite cannot observe. Dropping the two MssqlQuery.ts entries keeps the fixture coupling covered.
Check List
Issue partially addressed by this PR
#11826
Scope and limitations
Boolean dimension metadata does not distinguish BIT-valued SQL from predicate SQL. This PR therefore preserves raw references without guessing their representation or adding model configuration: