Conversation
…ess paths apache#68214 added retainPredicatePathsInFinalAllAccessPaths, which removes every predicate access path that is not literally one of the final all access paths. That is right for NULL/OFFSET paths, which are stripped from the all paths on purpose, but it also removes ordinary sub-field paths whenever the all paths collapse to the whole-column path: SELECT s FROM tbl WHERE struct_element(s, 'city') = 'x' -- all: [s], predicate: [] (was [s.city]) BE then cannot tell which sub-column the predicate reads, loses the eager/lazy split and reads the column as one unit. On branch-4.2 this shows up in test_iceberg_variant_read as FilteredRowsByLazyRead = 0 for SELECT CAST(v AS STRING) FROM variant_page_pruning WHERE CAST(v['n'] AS INT) > 3000 Only file scans hit this: on OLAP tables a variant sub-path predicate gets its own sub-column slot, which is why the existing tests did not catch it. Keep the NULL/OFFSET cleanup, since BE switches the whole iterator to NULL_MAP_ONLY/OFFSET_ONLY once such a path shows up. Any other predicate path is kept, and added to the all paths when no wider path covers it, as master's addPredicatePathsToFinalAllAccessPaths does.
Contributor
Author
|
run buildall |
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
hubgeter
marked this pull request as draft
September 21, 2026 09:37
Contributor
FE UT Coverage ReportIncrement line coverage |
hubgeter
marked this pull request as ready for review
September 21, 2026 16:36
Contributor
Author
|
/review |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: None
Related PR: #68214
Problem Summary:
retainPredicatePathsInFinalAllAccessPaths, added by #68214, removes every predicate access path that is not literally one of the final all access paths. That is what NULL/OFFSET paths need, because they are stripped from the all paths on purpose, but it also removes ordinary sub-field paths whenever the all paths collapse to the whole-column path:BE then cannot tell which sub-column the predicate reads, so it loses the eager/lazy split and reads the column as one unit. On branch-4.2 this shows up in
test_iceberg_variant_read, where the lazy-read check seesFilteredRowsByLazyRead = 0forOnly file scans hit this: on OLAP tables a variant sub-path predicate gets its own sub-column slot, so its all paths already contain the predicate path. That is why the existing unit tests did not catch it, and why the new test uses a struct column, which keeps one slot for the whole column.
This PR keeps the NULL/OFFSET cleanup, since BE switches the whole iterator to
NULL_MAP_ONLY/OFFSET_ONLYonce such a path shows up and skips the children. Every other predicate path is kept, and added to the all paths when no wider path covers it — the behaviour of master'saddPredicatePathsToFinalAllAccessPaths.Release note
Fix nested/variant sub-field predicates losing lazy materialization on branch-4.2 when the whole column is read.
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)