Skip to content

fix: MDEV-40846 DuckDB handles functionality that is based on invisible columns, e.g. WITH SYSTEM VERSIONING. - #5581

Merged
drrtuy merged 1 commit into
MariaDB:11.4from
drrtuy:11.4-MDEV-40846-inv-columns
Aug 21, 2026
Merged

fix: MDEV-40846 DuckDB handles functionality that is based on invisible columns, e.g. WITH SYSTEM VERSIONING.#5581
drrtuy merged 1 commit into
MariaDB:11.4from
drrtuy:11.4-MDEV-40846-inv-columns

Conversation

@drrtuy

@drrtuy drrtuy commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

I'll invoke both skills you referenced.

I'll now invoke the pr-message skill.

I'll gather the last commit info and diff.

Here's the PR message for the last commit (cc08036):


Fix MDEV-40846: handle invisible columns in DuckDB pushdown

What

Cross-engine queries touching invisible columns (e.g. tables WITH SYSTEM VERSIONING) produced incorrect results or column-count mismatches when pushed down to DuckDB. This PR makes the pushdown planner recognize invisible columns and fall back to MariaDB execution when it can't safely handle them.

Key changes

  • ha_duckdb_pushdown.cc: add Invisible_field_enumerator and query_references_invisible_fields() to bail out of pushdown when a query explicitly references invisible fields (excluding implicit versioning fields).
  • Add has_visible_field_prefix() so external tables only push down when invisible columns are strictly trailing.
  • Replace silent column-count truncation in next_row() with a hard error when DuckDB's result width doesn't match MariaDB metadata.
  • cross_engine_scan.cc: stop binding trailing invisible fields to the DuckDB scan.
  • Add cross_engine_join test cases (9–11) covering system versioning plus trailing/non-trailing invisible columns.

How to test

Run mysql-test suite duckdb.cross_engine_join.

@drrtuy
drrtuy force-pushed the 11.4-MDEV-40846-inv-columns branch from cc08036 to 566e626 Compare August 21, 2026 16:05
@drrtuy
drrtuy enabled auto-merge (rebase) August 21, 2026 16:07
@drrtuy
drrtuy merged commit 5af13b6 into MariaDB:11.4 Aug 21, 2026
16 of 18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant