Skip to content

split: fix overflow on a huge -n l/K/N chunk count - #14374

Merged
sylvestre merged 1 commit into
uutils:mainfrom
harshasiddartha:split-fix-kth-chunk-count-overflow
Sep 4, 2026
Merged

split: fix overflow on a huge -n l/K/N chunk count#14374
sylvestre merged 1 commit into
uutils:mainfrom
harshasiddartha:split-fix-kth-chunk-count-overflow

Conversation

@harshasiddartha

Copy link
Copy Markdown
Contributor

What was wrong

split -n l/K/N FILE panicked with attempt to add with overflow (exit 101) under overflow checks when N was very large:

$ head -c 100000 /dev/zero | tr '\0' a > big
$ split -n l/1/9999999999 big > /dev/null
thread 'main' panicked at src/uu/split/src/split.rs:1158:13:
attempt to add with overflow

A release build wrapped the counter silently instead, but still spent minutes spinning before it finished.

Root cause

n_chunks_by_line advances to the next chunk one chunk at a time. When the input is shorter than the requested number of chunks, every trailing chunk is zero-sized, so num_bytes_should_be_written stops growing and the loop condition stays true until chunk_number reaches num_chunks — up to N iterations. The count of chunks stepped over was let mut skipped = -1;, which is inferred as i32, so it overflowed once N passed i32::MAX.

Just widening the counter is not enough: N is parsed as a u64, so it can exceed i64::MAX, and the loop would still step through all N chunks — effectively forever for N near u64::MAX.

The fix

  • As soon as the loop reaches a zero-sized chunk, jump to the last chunk and stop. Every chunk after a zero-sized one is zero-sized too, so this reaches exactly the same chunk number that stepping one at a time would, without the iterations.
  • Drop the skipped counter and take the --elide-empty-files rollback from the chunk number captured before the loop (chunk_number.min(first_chunk_number + 1)). This is equivalent to the previous chunk_number -= skipped and leaves no counter that can overflow.

Output is unchanged: -n l/1/9999999999 on a short input now produces what -n l/1/3, -n l/1/100 and -n l/1/1000000 already produced, and the existing elide-empty-files behaviour is untouched.

How it was tested

  • Added test_number_by_lines_kth_huge_number_of_chunks in tests/by-util/test_split.rs. On main it fails (the run is killed by the 30s test timeout; run by hand the binary panics and exits 101), and it passes with this change.
  • The full split suite passes: cargo test --no-default-features --features split --test tests test_split → 119 passed, 0 failed.
  • Manually: split -n l/1/9999999999 big now exits 0 immediately with the full 100000 bytes, and split -n l/1/18446744073709551615 big works as well.
  • cargo clippy -p uu_split --all-targets is clean, cargo fmt reports no changes.

Fixes #14328

`split -n l/K/N` advances to the next chunk one chunk at a time. Once the
input is shorter than the number of chunks, the trailing chunks are all
zero-sized, so the loop makes no progress on the byte count and keeps
counting up to N. The counter for the chunks it stepped over was inferred
as `i32`, so a large N overflowed it and panicked under overflow checks.

Skip to the last chunk as soon as a zero-sized one is reached, since every
chunk after it is zero-sized too, and derive the number of chunks stepped
over from the chunk number itself so there is no separate counter to
overflow.
@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown

GNU testsuite comparison:

Skip an intermittent issue tests/pr/bounded-memory (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/tail/tail-n0f (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/date/resolution (passes in this run but fails in the 'main' branch)

@sylvestre
sylvestre merged commit 6ff289e into uutils:main Sep 4, 2026
103 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

split: arithmetic overflow (overflow-checks) on a huge -n l/K/N chunk count

2 participants