🔎 Search Terms
memory, oom, resolveObjectTypeMembers, getReducedType
🙁 Actual behavior
we have a ~37k file TS program at work and checking it on 7.0 needs ~13 GiB of heap single threaded (~16 GiB peak), and about double that with the default 4 checkers. people keep running out of memory, esp with the editor open too
i profiled it and there are two spots where the checker builds a ton of stuff it never uses:
- looking up one property on an instantiated class/interface (
expect(x).toBe, arr.map etc) instantiates every member of the type plus everything it inherits. on our program thats ~15.7M member symbols and only ~3M of them ever get used. isWeakType, getSingleSignature and friends also build the whole table just to count stuff
getReducedType builds the combined property for every name in an intersection just to check if it reduces to never. only a handful of names can actually do that (shared across constituents with a literal type, or private)
🙂 Expected behavior
only build what's needed. i have changes for both that give the same output as main. full go test suite passes (also with multiple checkers and -race) and i added tests for the edge cases i hit along the way
typescript-benchmarking tsc scenarios with both changes, 4 checkers, median of 3:
|
heap |
peak rss |
check time |
| vscode |
6.22 → 5.65 GiB (−9%) |
9.75 → 8.97 GiB (−8%) |
same |
| mui-docs |
4.75 → 2.72 GiB (−43%) |
6.80 → 4.03 GiB (−41%) |
5.89 → 3.86s (−35%) |
| xstate-main |
0.55 → 0.49 GiB (−10%) |
0.67 → 0.61 GiB (−8%) |
same |
| webpack |
0.94 → 0.84 GiB (−11%) |
1.15 → 1.00 GiB (−13%) |
0.51 → 0.48s (−6%) |
| Compiler |
−4% |
−6% |
same |
| Compiler-Unions |
−8% |
−2% |
same |
single threaded looks about the same (vscode −8% heap, mui-docs −35%). the intersection change is basically all of the mui-docs win and the member change is the ~8-10% on everything else. on our own program heap goes 13.1 → 10.4 GiB
output matched main on every run except TS6059 on mui-docs, but that one flakes on main too (#64477)
opened #64475 (members) and #64476 (intersections) as drafts. #64475 overlaps with #64372, and a lot of the complexity in it is there to keep the half built members behavior that #64372 removes. so if that lands first ill rebase and it should get a lot simpler
Additional information about the issue
--generateTrace couldnt handle a program this size (the types dump went past 160 GB before i killed it) so i found these with go heap profiles and some counters hacked into the checker
fwiw i used claude code for a lot of the digging and the code. ive reviewed it and ill be the one following up on review
🔎 Search Terms
memory, oom, resolveObjectTypeMembers, getReducedType
🙁 Actual behavior
we have a ~37k file TS program at work and checking it on 7.0 needs ~13 GiB of heap single threaded (~16 GiB peak), and about double that with the default 4 checkers. people keep running out of memory, esp with the editor open too
i profiled it and there are two spots where the checker builds a ton of stuff it never uses:
expect(x).toBe,arr.mapetc) instantiates every member of the type plus everything it inherits. on our program thats ~15.7M member symbols and only ~3M of them ever get used.isWeakType,getSingleSignatureand friends also build the whole table just to count stuffgetReducedTypebuilds the combined property for every name in an intersection just to check if it reduces tonever. only a handful of names can actually do that (shared across constituents with a literal type, or private)🙂 Expected behavior
only build what's needed. i have changes for both that give the same output as main. full go test suite passes (also with multiple checkers and
-race) and i added tests for the edge cases i hit along the waytypescript-benchmarking tsc scenarios with both changes, 4 checkers, median of 3:
single threaded looks about the same (vscode −8% heap, mui-docs −35%). the intersection change is basically all of the mui-docs win and the member change is the ~8-10% on everything else. on our own program heap goes 13.1 → 10.4 GiB
output matched main on every run except TS6059 on mui-docs, but that one flakes on main too (#64477)
opened #64475 (members) and #64476 (intersections) as drafts. #64475 overlaps with #64372, and a lot of the complexity in it is there to keep the half built members behavior that #64372 removes. so if that lands first ill rebase and it should get a lot simpler
Additional information about the issue
--generateTracecouldnt handle a program this size (the types dump went past 160 GB before i killed it) so i found these with go heap profiles and some counters hacked into the checkerfwiw i used claude code for a lot of the digging and the code. ive reviewed it and ill be the one following up on review