array_append / array_prepend / array_replace* discard the input list's inner field, contradicting their promised return type

[adriangb]

Describe the bug

array_append, array_prepend, array_replace, array_replace_n and array_replace_all have exactly the defect that #24341 describes for array_slice: return_type promises the input list type verbatim (inner field name, nullability and metadata included), but the kernel rebuilds the output's inner field from scratch with Field::new_list_field(..., true).

Promise sites:

• array_append — concat.rs:107-114 (Ok(array_type.clone()))
• array_prepend — concat.rs:189-196
• array_replace / array_replace_n / array_replace_all — replace.rs:121, :220, :329 (Ok(args[0].clone()))

Payload sites:

• general_append_and_prepend — concat.rs:568
• general_replace — replace.rs:505
• general_replace_with_scalar — replace.rs:601

As in #24341, this trips the return-type assertion from #17515 on debug builds, and on release builds silently yields a batch whose inner field name/nullability disagrees with the schema the planner recorded.

array_concat is not affected — it computes a fresh return type via type_union_resolution rather than cloning arg_types[0], so promise and payload agree. I also checked array_remove, array_distinct, array_union, array_intersect, array_sort and array_resize: all fine.

To Reproduce

Inner field name — via the Spark array function, whose list field is named element (spark_array.rs):

SELECT array_append(array(1, 2), 3);
SELECT array_prepend(0, array(1, 2));
SELECT array_replace(array(1, 2, 3), 2, 9);
SELECT array_replace_n(array(1, 2, 2), 2, 9, 1);
SELECT array_replace_all(array(1, 2, 2), 2, 9);

Internal error: Assertion failed: result_data_type == *expected_type: Function 'array_append'
returned value of type 'List(Int64)' while the following type was promised at planning time
and expected: 'List(Int64, field: 'element')'.

Non-nullable inner field:

select array_append(arrow_cast(make_array(1, 2), 'List(non-null Int32)'), 3);
select array_replace(arrow_cast(make_array(1, 2), 'List(non-null Int32)'), 2, 9);

Internal error: Assertion failed: result_data_type == *expected_type: Function 'array_append'
returned value of type 'List(Int64)' while the following type was promised at planning time
and expected: 'List(non-null Int64)'.

Expected behavior

The returned list's inner field matches the promised return type — with the caveat below.

Additional context

The field-name symptom is a 55.0.0 regression, same origin as #24341. I bisected it: at 5b22857036 ("fix: avoid extraneous casts for equivalent nested types", #20945, merged 2026-06-04) all five queries above fail; at its parent 467d2c3db8 all five succeed and return List(Int64). That commit is in 55.0.0-rc2 and not in 54.1.0. Before it, signature coercion inserted a cast normalizing List(element) → List(item), which made promise and payload agree by accident.

The non-nullable symptom is not a regression — it reproduces identically at 467d2c3db8.

These are not a pure copy of the #24345 fix. For array_slice it is sufficient to thread the input's FieldRef through, because slicing can only ever remove elements. For append/prepend/replace the new element can itself be null, so the promised type is wrong at the source:

select array_append(arrow_cast(make_array(1, 2), 'List(non-null Int32)'), NULL);
select array_replace(arrow_cast(make_array(1, 2), 'List(non-null Int32)'), 2, NULL);

Both promise List(non-null Int32) while the result genuinely contains a NULL element. Simply cloning the input field would produce an array arrow rejects with Non-nullable field of ListArray cannot contain nulls. So the fix needs return_type (or better, return_field_from_args) to carry the input field's name and metadata through while widening nullable when the appended/replacement argument is nullable, and the kernel to use that same field.

Found while reviewing #24345, which fixes the array_slice half of this in #24341. Worth deciding whether these should also land before 55.0.0 final, since they regressed in the same commit.

[adriangb]

Thanks for handling this @timsaucer !