Conversation
libcomet is loaded with dlopen, so its thread-locals use the general-dynamic TLS model and every access calls __tls_get_addr. track() made three accesses per allocation and per free (the re-entrancy flag twice, and the drift cell with its lazy-destructor state check), which cost about 5% on TPC-H Q21. Keep the drift and a phase in one const-initialized thread-local with no destructor, so the fast path is one access, a phase check and an add. The exit-time settle moves to a zero-sized guard thread-local that is registered once per thread, on its first tracked allocation, with the phase marking the re-entrant allocations registration can make. After the guard's destructor runs, deltas go straight to the shared balance. Closes apache#6165.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #6165.
Rationale for this change
libcomet.sois loaded by the JVM withdlopen, so itsthread_local!variables use the general-dynamic TLS model. Every access calls__tls_get_addrin the dynamic loader. The allocation accounting wrapper'strack()made three thread-local accesses on every allocation and every free:IN_TRACK.replace(true)LOCAL_DRIFT.try_with(...), which also checks the lazily registered destructor's stateIN_TRACK.set(false)With the wrapper installed, this cost about 5% on TPC-H SF100 Q21. The
alloc_overheadbenchmark does not show it, because it linkscometas an rlib into an executable, where a thread-local access is a single instruction.The wrapper is behind the
alloc-accountingfeature onmain. #6162 installs it in every build.What changes are included in this PR?
alloc_accounting.rsonly.const-initialized thread-local with no destructor. Tracking an allocation is now one thread-local access, a phase check and an add. No lazy-initialization check is needed, and the state stays readable while the thread's other thread-local destructors run.SettleOnExit, a zero-sized thread-local whose destructor does the settle. It is registered once per thread, on the thread's first tracked allocation, in theregisteringphase. Registering a destructor can allocate on some platforms, and that allocation re-enterstrackand goes straight to the shared balance.exited, and deltas from later destructors go straight to the shared balance rather than to a drift nothing would settle.How are these changes tested?
Unit tests, run in the default build, with
alloc-accounting, and withjemalloc,alloc-accounting:registeringandexitedphases bypass the drift and reach the shared balance.TPC-H SF100 Q21 on a 32-core Linux box (Spark 4.1.1, 2 executors x 8 cores, glibc allocator). These builds have the wrapper installed by default, as in #6162, with this change applied on top. Each build ran 3 times, alternating, with 3 iterations per run. The table uses the median of iterations 2 and 3.
All builds returned the same result.
perfover the same query, summed over both executors:__rust_alloc/__rust_deallocself__tls_get_addrThe remaining cost is the one
__tls_get_addrper call, andalloccalling the backend and accounting on return instead of tail-calling it. The shared atomic is not a factor: itslock addinstructions account for well under 0.1% of samples. Removing the remaining lookup would need the initial-exec TLS model, which is nightly-only and unsafe for adlopened library, or a design without thread-locals.