Skip to content

[fix](agg) Align complex aggregate null ordering #67439

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Mryange merged 3 commits into from
Sep 15, 2026
+14 −11
Merged

[fix](agg) Align complex aggregate null ordering #67439

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
000770f
[fix](agg) Align complex aggregate null ordering
Mryange Sep 2, 2026
ca9b1f9
upd case
Mryange Sep 7, 2026
b46c17a
[test](regression) Align complex aggregate expected results
Mryange Sep 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions be/src/exprs/aggregate/aggregate_function_min_max.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -616,7 +616,7 @@ struct SingleValueDataComplexType {
}

bool change_if_less(const IColumn& column, size_t row_num, Arena& arena) {
if (!has() || column_data->compare_at(0, row_num, column, -1) == 1) {
if (!has() || column_data->compare_at(0, row_num, column, 1) == 1) {

@github-actions github-actions Bot Sep 2, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SingleValueDataComplexType is also the state for unary complex min/max, not just the *_by key. With this +1, nested ColumnNullable treats NULL as greatest, so [10,NULL] beats [10,5] for max(arr) and loses for min(arr). That conflicts with Doris' documented ARRAY aggregate order (NULL is smallest) and the unchanged maxmin_array_2 expectation. Please keep the aggregate comparator policy consistent with the documented contract (or scope a different hint to the intended *_by path) and add/update plain, merge, and window coverage.

change(column, row_num, arena);
return true;
} else {
Expand All @@ -625,7 +625,7 @@ struct SingleValueDataComplexType {
}

bool change_if_less(const Self& to, Arena& arena) {
if (to.has() && (!has() || column_data->compare_at(0, 0, *to.column_data, -1) == 1)) {
if (to.has() && (!has() || column_data->compare_at(0, 0, *to.column_data, 1) == 1)) {

@github-actions github-actions Bot Sep 13, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Version the semantic change before merging old aggregate states

This overload merges an already-reduced complex state, but the PR reverses the order used to choose that sole state without creating a compatibility boundary. Both the base and head still advertise BE exec version 13. For example, an old v13 worker computing max over [1, NULL] and [1, 5] serializes only [1, 5]; under this PR v13 max should return [1, NULL], and a new BE merging the old state cannot recover the discarded row. Stored max_state/min_state values (and ARRAY-key *_by states) are likewise accepted because these functions have no compatibility-map entry. Please version/select the old policy for old states, or reject and require rebuilding them, and cover mixed/persisted merges.

@Mryange Mryange Sep 15, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个过去就是错的，fix不考虑兼容了

change(to, arena);
return true;
} else {
Expand All @@ -634,7 +634,7 @@ struct SingleValueDataComplexType {
}

bool change_if_greater(const IColumn& column, size_t row_num, Arena& arena) {
if (!has() || column_data->compare_at(0, row_num, column, -1) == -1) {
if (!has() || column_data->compare_at(0, row_num, column, 1) == -1) {
change(column, row_num, arena);
return true;
} else {
Expand All @@ -643,7 +643,7 @@ struct SingleValueDataComplexType {
}

bool change_if_greater(const Self& to, Arena& arena) {
if (to.has() && (!has() || column_data->compare_at(0, 0, *to.column_data, -1) == -1)) {
if (to.has() && (!has() || column_data->compare_at(0, 0, *to.column_data, 1) == -1)) {
change(to, arena);
return true;
} else {
Expand All @@ -660,7 +660,7 @@ struct SingleValueDataComplexType {
type == TYPE_AGG_STATE) {
return false;
} else {
return !column_data->compare_at(0, row_num, column, -1);
return !column_data->compare_at(0, row_num, column, 1);
}
}

Expand Down
14 changes: 8 additions & 6 deletions ...on-test/data/query_p0/sql_functions/aggregate_functions/test_aggregate_all_functions2.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -389,16 +389,19 @@ true
[1, 2, 5] [1, 2]
[11, 22, 33, 44] [3, 1]
[10] []
[11, null, null, 55] [1, null, null, 4]
[11, null, null, 55] [1, 2, 3, 4]

-- !maxminby_array_2 --
[1, 2, 5] [11, 22, 33, 44]

-- !maxminby_array_3 --
{"A":10, "B":1} {"x":50, "y":60}
{"A":10, "B":5} {"x":50, "y":60}

-- !maxminby_array_4 --
{"a":10, "b":"tt"} {"a":4, "b":"delta"}
{"a":10, "b":"ten"} {"a":4, "b":"delta"}

-- !maxminby_array_null_order --
[11, null, null, 55] [1, 2, 3, 4]

-- !maxminby_array_5 --
1 [5, 6] [7]
Expand All @@ -422,7 +425,7 @@ true
2 {"foo":1, "bar":2} {"foo":2, "bar":1}
3 {"A":10, "B":1} {"key1":99, "key2":98}
4 {"A":5, "B":10} {"x":50, "y":60}
5 {"A":10, "B":5} {"A":null, "B":null}
5 {"A":10, "B":5} {"A":null, "B":5}

-- !maxminby_struct_2 --
{"a":5, "b":"echo"} {"a":10, "b":"tt"}
Expand All @@ -439,7 +442,7 @@ true
2 {"a":5, "b":"echo"} {"a":6, "b":"zulu"}
3 {"a":10, "b":"tt"} {"a":8, "b":"eight"}
4 {"a":9, "b":"nine"} {"a":4, "b":"delta"}
5 {"a":10, "b":"ten"} {"a":null, "b":null}
5 {"a":10, "b":"ten"} {"a":null, "b":"ten"}

-- !maxmin_array_3 --
[[3, 4], [3, 4]] [[1, 2], [3, 4]]
Expand All @@ -449,4 +452,3 @@ true

-- !maxminby_array_7 --
[[3, 4], [3, 4]] [[1, 2], [3, 4]]

1 change: 1 addition & 0 deletions ...st/suites/query_p0/sql_functions/aggregate_functions/test_aggregate_all_functions2.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -336,6 +336,7 @@ suite("test_aggregate_all_functions2") {
qt_maxminby_array_2 """SELECT max_by(arr, weight), min_by(arr, weight) from test_maxmin"""
qt_maxminby_array_3 """SELECT max_by(mp, arr), min_by(mp, arr) from test_maxmin"""
qt_maxminby_array_4 """SELECT max_by(st, arr), min_by(st, arr) from test_maxmin"""
qt_maxminby_array_null_order """SELECT max_by(arr, arr), min_by(arr, arr) from test_maxmin where id = 5"""

qt_maxminby_array_5 """SELECT id, max_by(arr, weight), min_by(arr, weight) from test_maxmin group by id order by id"""

Expand Down
Loading