PyObject_Realloc(NULL, size) crashes free-threaded 32-bit Python 3.15.0b2

[godlygeek]

Crash report

What happened?

I haven't managed to reproduce this with a build of CPython from source, but it reproduces using the manylinux images shipped by the PyPA on an x86-64 machine:

docker run -it quay.io/pypa/manylinux_2_34_i686 /usr/local/bin/python3.15t -c 'import ctypes; PyObject_Realloc = ctypes.pythonapi.PyObject_Realloc; PyObject_Realloc.argtypes = [ctypes.c_void_p, ctypes.c_size_t]; PyObject_Realloc.restype = ctypes.c_void_p; print(PyObject_Realloc(None, 984))'

It only crashes for python3.15t, not for python3.15 or python3.14t.

It only crashes for the i686 image, not the x86-64 image.

(gdb) bt
#0  0x5ebd5a4e in ?? ()
#1  0x5ebd7a21 in PyObject_Realloc ()
#2  0xf2cc37dc in ffi_call_i386 () at ../src/x86/sysv.S:121
...

(gdb) x/30i 0x5ebd5a4e - 50
   0x5ebd5a1c:  nopl   0x0(%eax)
   0x5ebd5a20:  mov    0xc(%ebp),%edi
   0x5ebd5a23:  mov    (%edi,%edx,1),%edi
   0x5ebd5a26:  mov    %edi,(%esi,%edx,1)
   0x5ebd5a29:  mov    0xc(%ebp),%edi
   0x5ebd5a2c:  add    $0x4,%edx
   0x5ebd5a2f:  cmp    %edx,%eax
   0x5ebd5a31:  jne    0x5ebd5a20
   0x5ebd5a33:  lea    (%esi,%eax,1),%edx
   0x5ebd5a36:  add    %edi,%eax
   0x5ebd5a38:  sub    $0x4,%esp
   0x5ebd5a3b:  push   %ecx
   0x5ebd5a3c:  push   %eax
   0x5ebd5a3d:  push   %edx
   0x5ebd5a3e:  call   0x5edd6220 <memcpy@plt>
   0x5ebd5a43:  add    $0x10,%esp
   0x5ebd5a46:  lea    -0x1(%edi),%eax
   0x5ebd5a49:  and    $0xffc00000,%eax
=> 0x5ebd5a4e:  mov    0x6c(%eax),%edx
   0x5ebd5a51:  sub    %eax,%edi
   0x5ebd5a53:  shr    $0x9,%edi
   0x5ebd5a56:  and    $0xffffffc0,%edi
   0x5ebd5a59:  lea    (%eax,%edi,1),%ecx
   0x5ebd5a5c:  sub    0x74(%eax,%edi,1),%ecx
   0x5ebd5a60:  mov    %gs:0x0,%edi
   0x5ebd5a67:  add    $0x70,%ecx
   0x5ebd5a6a:  cmp    %edx,%edi
   0x5ebd5a6c:  jne    0x5ebd5acf
   0x5ebd5a6e:  cmpb   $0x0,0xe(%ecx)
   0x5ebd5a72:  jne    0x5ebd5ad6

It looks like it's crashing after the memcpy call that realloc makes to copy from the old buffer to the new. The only other thing _PyObject_MiRealloc does after that point is call mi_free(ptr) to free the old pointer, but that should be special casing null...

The crash is from trying to access:

(gdb) p $_siginfo._sifields._sigfault.si_addr
$1 = (void *) 0xffc0006c

The address's high byte being 0xFF on a 32-bit address space makes me think something has wrapped around by subtracting from the null pointer to find the malloc housekeeping structures or something, but I haven't figured out why, since all of the code does seem to be properly guarded against null pointers.

CPython versions tested on:

3.15

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.15.0b2 free-threading build (main, Jun 6 2026, 08:05:01) [Clang 22.1.7 ]

Linked PRs

• gh-151297: Fix undefined behavior in _PyObject_MiRealloc #151358
• [3.15] gh-151297: Fix undefined behavior in _PyObject_MiRealloc (GH-151358) #151388

[godlygeek]

Ooh, it also seems to be intermittent:

[root@bacf47817bc8 /]# /usr/local/bin/python3.15t -c 'import ctypes; PyObject_Realloc = ctypes.pythonapi.PyObject_Realloc; PyObject_Realloc.argtypes = [ctypes.c_void_p, ctypes.c_size_t]; PyObject_Realloc.restype = ctypes.c_void_p; print(PyObject_Realloc(None, 984))'
Segmentation fault (core dumped)
[root@bacf47817bc8 /]# /usr/local/bin/python3.15t -c 'import ctypes; PyObject_Realloc = ctypes.pythonapi.PyObject_Realloc; PyObject_Realloc.argtypes = [ctypes.c_void_p, ctypes.c_size_t]; PyObject_Realloc.restype = ctypes.c_void_p; print(PyObject_Realloc(None, 984))'
3777305600
[root@bacf47817bc8 /]# /usr/local/bin/python3.15t -c 'import ctypes; PyObject_Realloc = ctypes.pythonapi.PyObject_Realloc; PyObject_Realloc.argtypes = [ctypes.c_void_p, ctypes.c_size_t]; PyObject_Realloc.restype = ctypes.c_void_p; print(PyObject_Realloc(None, 984))'
3911523328
[root@bacf47817bc8 /]# /usr/local/bin/python3.15t -c 'import ctypes; PyObject_Realloc = ctypes.pythonapi.PyObject_Realloc; PyObject_Realloc.argtypes = [ctypes.c_void_p, ctypes.c_size_t]; PyObject_Realloc.restype = ctypes.c_void_p; print(PyObject_Realloc(None, 984))'
Segmentation fault (core dumped)

[sergey-miryanov]

Using ctypes to achieve non-standard behavior is a bit too hacky and falls outside our project's scope.

Furthermore, Py_None is a statically allocated object, so calling PyObject_Realloc on it results in undefined behavior.

Given this, I'm not sure we should try to do anything about it here.

[godlygeek]

ctypes isn't relevant here, you can reproduce this with a C extension if you prefer. I just figured a reproducer that doesn't require compiling extra code would be easier to work with.

This does not try to reallocate Py_None. When a ctypes function that expects a pointer is passed None, the FFI layer passes NULL down to the called function.

[godlygeek]

I've just figured out how to reproduce the crash with a debug build: it happens with Clang but not with GCC!

It's possible this is a compiler bug rather than a CPython bug...

mi_free (p=0x0) at Objects/mimalloc/alloc.c:576
576       const bool          is_local= (_mi_prim_thread_id() == mi_atomic_load_relaxed(&segment->thread_id));
(gdb) bt
#0  mi_free (p=0x0) at Objects/mimalloc/alloc.c:576
#1  _PyObject_MiRealloc (ctx=<optimized out>, ptr=0x0, nbytes=984) at Objects/obmalloc.c:368
#2  0x08135e40 in PyObject_Realloc (ptr=0x0, new_size=984) at Objects/obmalloc.c:1728
#3  0xf5c187dc in ffi_call_i386 () at ../src/x86/sysv.S:121

mi_free (p=0x0) at Objects/mimalloc/alloc.c:576
576       const bool          is_local= (_mi_prim_thread_id() == mi_atomic_load_relaxed(&segment->thread_id));

(gdb) list
571     // fast path written carefully to prevent spilling on the stack
572     void mi_free(void* p) mi_attr_noexcept
573     {
574       if mi_unlikely(p == NULL) return;
575       mi_segment_t* const segment = mi_checked_ptr_segment(p,"mi_free");
576       const bool          is_local= (_mi_prim_thread_id() == mi_atomic_load_relaxed(&segment->thread_id));
577       mi_page_t* const    page    = _mi_segment_page_of(segment, p);
578
579       if mi_likely(is_local) {                       // thread-local free?
580         if mi_likely(page->flags.full_aligned == 0)  // and it is not a full page (full pages need to move from the full bin), nor has aligned blocks (aligned blocks need to be unaligned)
(gdb) p p
$1 = (void *) 0x0

Somehow, despite p being NULL, the Clang-compiled binary has wound up on line 576 of mi_free, despite that function being gated with if mi_unlikely(p == NULL) return;

Either this is a compiler bug, or somewhere we performed an operation on p that would be UB if performed on a NULL pointer and so the compiler has branched assuming that p must not be NULL...

[sergey-miryanov]

Thanks for clarification.

[ZeroIntensity]

Yeah, this feels a lot like a compiler bug. That mi_unlikely might be doing something suspicious when aggressively compiled by Clang. I would try removing that and seeing if the crash magically disappears.

I've just figured out how to reproduce the crash with a debug build: it happens with Clang but not with GCC!

Is this still i686? I've been trying to reproduce this on x86-64, but I haven't been able to.

[godlygeek]

Yes, still needs i686. I've been reproducing it in a quay.io/pypa/manylinux_2_34_i686 container, run from a Linux x86-64 machine.

[godlygeek]

Specifically I got it reproducing with:

docker run -v $PWD:/workspace -it quay.io/pypa/manylinux_2_34_i686

and then

yum install clang
cd /tmp
git clone https://github.com/python/cpython
cd cpython
CC=clang CFLAGS_NODIST='-g -O2 -D_FORTIFY_SOURCE=2' ./configure --disable-shared --disable-gil
make -j
gdb --args ./python -c 'import ctypes; PyObject_Realloc = ctypes.pythonapi.PyObject_Realloc; PyObject_Realloc.argtypes = [ctypes.c_void_p, ctypes.c_size_t]; PyObject_Realloc.restype = ctypes.c_void_p; print(PyObject_Realloc(None, 984))'