Skip to content

Make the C extension Ractor-safe by removing msgpack_rmem_* slab allocator#392

Open
ianks wants to merge 1 commit into
msgpack:masterfrom
ianks:ractor-safe-rmem
Open

Make the C extension Ractor-safe by removing msgpack_rmem_* slab allocator#392
ianks wants to merge 1 commit into
msgpack:masterfrom
ianks:ractor-safe-rmem

Conversation

@ianks

@ianks ianks commented Jun 10, 2026

Copy link
Copy Markdown

This makes the extension safe to use from a non-main Ractor. The notable change here is the removal of the msgpack_rmem_* routines, which serve as a mechanism for efficiently providing chunks of memory for decoding work.

Today the page-recycling slab is a process-global msgpack_rmem_t mutated through an unsynchronized bitmask, so parallel Ractors (or threads) race on it.

The fix drops the global slab and serves pages with xmalloc/xfree, since modern mallocs (i.e. jemalloc) are good at recycling anyway.

Details / Research

Single-threaded, jemalloc 5.3, decode-heavy workload, so treat these as directional.

Throughput

No measurable impact at the request level. On a realistic HTTP-request replay (paired by recording, QoS-pinned, ~9.4k samples/variant) the warm path is a wash and allocations are identical:

warm, paired (this PR vs slab) Δ 95% CI
wall time −0.1% −0.8% to +4.9%
CPU −0.3% −1.9% to +3.9%
allocations identical

Dropping the recycling costs ~3% throughput (and more GC churn) on an isolated decode microbench, but msgpack is well under 2% of a real request and the warm path doesn't decode at all, so it doesn't surface end-to-end.

RSS Impact

Memory usage seems fine as well, with the bare-xmalloc (this PR) having a higher peak as decay time increases (this is expected, and not a problem).

sustained decode slab (today) xmalloc (this PR)
dirty_decay_ms:10000 (default) 43 MiB 102 MiB
dirty_decay_ms:1000 33 MiB 34 MiB
peak (live working set) ~113 MiB ~114 MiB

closes #390

@ianks ianks force-pushed the ractor-safe-rmem branch from cc53acc to d1c9fce Compare June 10, 2026 20:43
@ianks ianks changed the title Make the C extension Ractor-safe Make the C extension Ractor-safe by removing msgpack_rmem_* slab allocator Jun 10, 2026
@ianks ianks marked this pull request as ready for review June 10, 2026 22:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ractor incompatibility: DefaultFactory not shareable, C extension methods marked ractor-unsafe

1 participant