perf(flow-php/etl): native int/float fast-path in sum() with opt-in exact mode#2522
Merged
Merged
Conversation
…xact mode
Sum::aggregate() routed every addition through brick/math: a new
Calculator instance per row, two string casts and two BigDecimal
parses per addition. On a 1M-row aggregation that dominated the cost
of the most common aggregating function.
- sum() now accumulates through native int/float arithmetic and
mirrors Calculator::add() by returning int whenever the running
sum has no fractional part
- sum(ref('x'), exact: true) opts back into arbitrary-precision
decimal arithmetic
- a shared Calculator instance lives in Config and is exposed through
FlowContext::calculator(); Sum (exact mode) and Average pull it
from the context instead of constructing one per row
Sum::aggregate() micro-benchmark, 1M rows: int 3.57 -> 0.45 us/row
(8x), float 5.07 -> 0.51 us/row (10x); exact mode keeps the previous
characteristics.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## 1.x #2522 +/- ##
=========================================
Coverage 85.97% 85.97%
- Complexity 22261 22271 +10
=========================================
Files 1663 1663
Lines 68418 68436 +18
=========================================
+ Hits 58822 58840 +18
Misses 9596 9596 🚀 New features to boost your workflow:
|
norberttech
added
the
hackaton-07-2026
The exact flag follows the ScalarFunction|bool parameter convention: it accepts a plain boolean, lit(...) or any scalar expression evaluated per row (e.g. a boolean column). Plain booleans keep the allocation-free fast path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Sum::aggregate()routed every addition through brick/math: anew Calculator()instance per row, thenBigDecimal::of((string) $a)->plus(BigDecimal::of((string) $b))— two string casts and two decimal parses per aggregated row, plus a fractional-part check and atoInt()/toFloat()conversion. On a 1M-row aggregation this dominated the cost of the most common aggregating function. Found while profiling pipelines against a 1M-row dataset.Changes
sum()accumulates through native int/float arithmetic by default and mirrors theCalculator::add()contract by returningintwhenever the running sum has no fractional part (sosumof[2.5, 2.5]still produces anint_entry(5)— guarded by a test),sum(ref('x'), exact: true)opts back into the previous arbitrary-precision decimal path — for decimal-sensitive use cases; the flag follows theScalarFunction|boolparameter convention, so it also takeslit(...)or any scalar expression evaluated per row (e.g. a boolean column) — plain booleans keep the allocation-free path,Calculatorinstance now lives inConfigand is exposed throughFlowContext::calculator()(same pattern asentryFactory());Sum(exact mode) andAverage(per-rowaggregate()/apply()) pull it from the context instead of constructing a new instance per row,FlowContext, fast-path vs exact-mode behavior on both the aggregation and the window-function path, and an anchor test pinning the int-when-whole result contract.Precision note: with the default fast path, float summation is plain IEEE-754 arithmetic. Through
result()this is not observable for typical sums —float_entry()already normalizes values viaBigDecimal::of((string) $value)at PHP's defaultprecision=14— and the raw accumulated value only surfaces throughWindowFunction::apply().exact: truekeeps decimal-exact accumulation end to end.Benchmarks
1M rows, PHP 8.3 + opcache, each run in a fresh process. Micro = 1M ×
Sum::aggregate()on a prebuiltRow; e2e =data_frame()->read(from_csv(...))->aggregate(sum(ref('seller_id')))->run()on a 57 MB / 1M-row CSV.1.xexact: trueThe modest e2e delta is expected: the pipeline cost is dominated by
Row/Entryconstruction (a separate concern); the sum's own share of the run drops from ~1.2 s to ~0.1 s. Peak memory unchanged (12 MB).Resolves: no related roadmap item — standalone performance fix, happy to open an issue if preferred
Change Log
Added
Fixed
Changed
Removed
Deprecated
Security