Speed up enum-case removal, finite type counting, large unions and loop widening of shapes - #6443
Speed up enum-case removal, finite type counting, large unions and loop widening of shapes#6443ondrejmirtes wants to merge 7 commits into
Conversation
|
You've opened the pull request against the latest branch 2.3.x. PHPStan 2.3 is not going to be released for months. If your code is relevant on 2.2.x and you want it to be released sooner, please rebase your pull request and change its target to 2.2.x. |
| if (count($allowedEnumCases) !== 1) { | ||
| continue; | ||
| } | ||
| $allowedEnumCaseKeys[$allowedEnumCases[0]->getClassName() . '::' . $allowedEnumCases[0]->getEnumCaseName()] = $key; |
There was a problem hiding this comment.
this code reads similar to what FiniteTypeSet is doing. maybe we should move the logic into FiniteTypeSet?
There was a problem hiding this comment.
Good call - done in b7fab07: the allowed subtypes go through FiniteTypeSet::create() and subtracted cases are matched with FiniteTypeSet::key(); the allowed subtypes it cannot key (a sealed class hierarchy) keep the equals() sweep. Re-measured on the file this was done for (Slevomat ImportAirportsCommand.php, 250-case enum loop): numbers in the PR description update.
ConstantArrayType::getFiniteTypes() forked a builder per optional key and only noticed after each key that the partial set had grown past CALCULATE_SCALARS_LIMIT, so a shape with many optional keys paid hundreds of builder clones per call for an empty result. Every caller that asks a large shape (the remove() finite-set difference, the ternary-arm and re-assignment checks in AssignHandler) hit this. Multiply the per-key counts first and only build when the product stays within the limit; the result is unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GvDxWpzXzticTTm5eLCsb2
TypeCombinator::remove() removes a union from a sealed hierarchy at once only when the object type is a supertype of the whole union. As soon as one member is already subtracted (`$a !== $b` narrowing inside a loop over Enum::cases() gets there quickly), the check fails and the members are peeled off one at a time - every peel rebuilds and re-describes the growing subtracted union, so removing 250 cases costs O(cases^2) describes. ObjectType::tryRemove() now drops the members the type no longer holds and subtracts the rest in one go, which is the same set difference. The allowed subtypes go through FiniteTypeSet: subtracted enum cases are matched by FiniteTypeSet::key() (a lookup per case) instead of an equals() sweep over every case, and the allowed subtypes it cannot key (a sealed class hierarchy) keep the sweep. ClassReflection::getAllowedSubTypes() memoises its result instead of rebuilding every case object on each call. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DfQHLTyyoMfjfa4sXJCGa5
…ion() A member passed to union() more than once contributes nothing, but the pairwise comparison still paid for every repeat - and array_values() of a shape with optional keys unions the same value type once per slot, so a 250-entry map built in a loop over an enum cost 10 s in one call. The repeats are dropped by identity before anything else. The flattening loop also rebuilt the argument list with array_splice() once per union member, quadratic in the number of members; it is a single pass now. array_merge() hands the first array back as it is when the second one is empty, so the keys the bucketing unset are renumbered inside the merge argument. build/spl-autoload-functions-pre-php-7.neon only ignored the "array_values is already a list" report that the old standalone renumbering produced on PHP < 8 (the array_merge() stub returns a list there); nothing reports it any more, so the file and its include go. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DfQHLTyyoMfjfa4sXJCGa5
… it holds union(U, x) rebuilt U's members whenever x was a constant scalar U already held: every member was re-bucketed by its description and a new union was assembled. ArrayType::setOffsetValueType() unions the key type with the offset for every offset written, so a rule checking a write to an array keyed by a union of a few hundred locale strings paid that once per member of the offset type. The union is now handed back untouched when it is a supertype of the scalar - the same members the general path produces. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GvDxWpzXzticTTm5eLCsb2
… shape
ArrayType::setExistingOffsetValueType() merged a written constant array
into the item shape all-or-nothing: once with every key required and once
with every optional key unset. A written shape whose optional keys can be
present independently (a nested `$a[$i][$j][$k]['x'] = ...` write in a loop)
then came out as `array{}|array{k1: T, ..., kN: T}`, which is unsound for
a value holding only some of the keys. Optional keys are now written as
optional: an existing key keeps its certainty and unions the value, a
missing one is added as optional.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DfQHLTyyoMfjfa4sXJCGa5
3f07ce0 to
0f06587
Compare
…ning MutatingScope::generalizeType() flattened both sides with TypeUtils::flattenTypes(), which turns a constant array with N optional keys into its 2^N concrete variants (four lossy representatives above ten) - only for the union right after to merge them back into the same shape, paying the pairwise reduceArrays() pass for hundreds of variants on every loop iteration. A method building a shape from a dozen conditional writes spent 1.8 s there. The per-key widening reads keys, values and optionality straight from the shape, so only unions are split now. The bug-13637 fixture keeps the loop's int<0, 8> bound on the middle key (the comment had called the previous `int` a residual) and describes the innermost level as the optional-key shape it is. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DfQHLTyyoMfjfa4sXJCGa5
76f47dd dropped the PHP 8.5 requirement from the three NoDiscard rule tests, but their fixtures still use the pipe operator and the (void) cast, which are PHP 8.5 syntax - on the PHP 7.4 job the pipe token cannot even be lexed. Each fixture is split: the plain calls stay in the original file and run on every PHP with attributes (7.4 reads #[\NoDiscard] as a comment, so nothing would be reported there), the (void) casts and pipes move to a -php85 fixture behind #[RequiresPhp('>= 8.5.0')]. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DfQHLTyyoMfjfa4sXJCGa5
0f06587 to
c971e68
Compare
Follow-up to #6442, from per-file
--debug -vvvcomparisons on Slevomat (2.2.x, pre-regression 93c49a1 and 2.3.x), tempest-framework, WordPress core and ShipMonk, with the remaining slow files traced through backtrace/timing instrumentation. Six commits, each measured on the files that exposed them.1. Removing a union of enum cases peeled one case at a time
TypeCombinator::remove()removes a union from a sealed hierarchy at once only when the object type is a supertype of the whole union. As soon as one member is already subtracted ($a !== $bnarrowing inside a loop overEnum::cases()gets there quickly) the check fails anddoRemove()peels the members off one at a time; every peel builds a newObjectTypewith a growing subtracted union and re-describes it, so removing 250 cases costs O(cases²) describes.ObjectType::tryRemove()now drops the members the type no longer holds (no-ops) and subtracts the rest in one go - the same set difference.matchAllowedSubTypes()matches subtracted enum cases throughFiniteTypeSet(value-identity keys, a lookup per case) instead of anequals()sweep over every allowed case per subtracted case; allowed subtypes it cannot key (a sealed class hierarchy) keep the sweep. Re-measured after the review change onImportAirportsCommand.php, three alternating rounds: 93c49a1 1.33 / 1.23 / 1.19 s, by-name index 0.99 / 0.94 / 0.96 s, FiniteTypeSet 1.02 / 0.97 / 0.96 s - the same gain.ClassReflection::getAllowedSubTypes()memoises its result instead of rebuilding every case object on each call.2.
ConstantArrayType::getFiniteTypes()built partial arrays it then threw awayThe builder was forked per optional key and only after each key was the partial count checked against
CALCULATE_SCALARS_LIMIT, so a shape with many optional keys paid hundreds of builder clones per call for an empty result. The per-key counts are multiplied first and the arrays are only built when the product stays within the limit; the result is unchanged.3.
TypeCombinator::doUnion()paid for repeated members and flattened quadraticallyA member passed to
union()more than once contributes nothing, but the pairwise comparison still paid for every repeat -array_values()of a shape with optional keys unions the same value type once per slot, andUnionTypeMethodReflection::getVariants()unions the same types across many members. Repeats are dropped by identity first. The flattening loop also rebuilt the argument list witharray_splice()once per union member, quadratic in the member count; it is a single pass now. (array_merge()returns the first array untouched when the second is empty, so the later renumbering stays - asarray_values()inside the merge.)4.
union(U, x)rebuilt U when x was a constant scalar U already heldEvery member was re-bucketed by its description and a new union assembled.
ArrayType::setOffsetValueType()unions the key type with the offset for every offset written, so a rule checking a write to an array keyed by a union of a few hundred locale strings paid that once per member of the offset type. The union is now handed back untouched when it is a supertype of the scalar - the same members the general path produces.5. Loop widening expanded optional-key shapes into every variant
MutatingScope::generalizeType()flattened both sides withTypeUtils::flattenTypes(), which turns a constant array with N optional keys into its 2^N concrete variants (four lossy representatives above ten) - only for the union right after to merge them back into the same shape, paying the pairwisereduceArrays()pass for hundreds of variants on every loop iteration. A method building a shape from a dozen conditional writes spent 1.8 s there. The per-key widening reads keys, values and optionality straight from the shape, so only unions are split now.6. A written shape's optional keys were merged all-or-nothing into a general array's item shape
Found while validating 5:
ArrayType::setExistingOffsetValueType()merged a written constant array into the item shape once with every key required and once with every optional key unset, so a nested$a[$i][$j][$k]['x'] = ...write in a loop came out asarray{}|array{k1: T, ..., kN: T}- unsound for a value holding only some of the keys. The old degraded widening hid it. Optional keys are now written as optional: an existing key keeps its certainty and unions the value, a missing one is added as optional. Thebug-13637fixture now keeps the loop'sint<0, 8>bound on the middle key (its comment had called the previousinta residual) and describes the innermost level as the optional-key shape it is.Measurements (
--debug -vvv"took", alternating runs; all files also slow on 2.2.x)app/ui/User/UpdateNonreadersCommand.php(432 LoC)app/ui/ProductList/TravelPresenter.php(onearray_values()of a 250-entry map)app/services/Search/Llm/Training/TrainingDataAnalysis.php(shape from ~10 conditional writes, widened in loops)app/ui/Airport/ImportAirportsCommand.php(250-case enum loop)app/services/Elasticsearch/Mapping/ElasticsearchMapperProductExtended.phppackages/intl/src/Catalog/CatalogInitializer.php(47 LoC, array keyed by ~600 locale strings)Whole projects, single process, sequential runs: tempest 50.9 s / 56.5 s vs 58.9 s / 64.9 s for 93c49a1 (spread over many files, none slower when re-timed alternating); Slevomat 510 s vs 2.2.x 519.6 s. Full Slevomat run through the phing target (locally built PHAR, fork + turbo), interleaved pairs: 586 s / 598 s vs 607 s / 602 s user CPU for the union change against the branch's previous commit; the later commits are neutral there. Error output identical on Slevomat, ShipMonk, tempest and WordPress core for every commit; no file slower in the per-file comparisons once re-timed alternating.
tests/PHPStan/Type,tests/PHPStan/Analyser,tests/PHPStan/Rulesgreen; phpcs and the full self-analysis clean.🤖 Generated with Claude Code
https://claude.ai/code/session_01DfQHLTyyoMfjfa4sXJCGa5