Windows: how to override malloc / free / new / delete globally ?

[jcelerier]

The simple following program crashes if linking against snmallocshim-static (MSYS2, with clang64 but reproducible on any other MinGW environment):

main.cpp:

#include <QObject>

int main(int argc, char *argv[]) {
    delete new QObject;
    return 0;
}

$ clang++ main.cpp  \
   -I/clang64/include/qt6/ \
   -I/clang64/include/qt6/QtCore \
   -std=c++20 \
    /clang64/lib/libQt6Core.dll.a \
    libsnmallocshim-static.a \
   -lbcrypt -rdynamic

$ lldb ./a.exe
(lldb) r
Process 19300 stopped
* thread #1, stop reason = Exception 0x80000003 encountered at address 0x7ffbd2e9ca82
    frame #0: 0x00007ffbd2e9ca83 ntdll.dll`RtlIsZeroMemory + 163
ntdll.dll`RtlIsZeroMemory:
->  0x7ffbd2e9ca83 <+163>: jmp    0x7ffbd2e9ca91 ; <+177>
    0x7ffbd2e9ca85 <+165>: movq   0xf8(%rsp), %rdi
    0x7ffbd2e9ca8d <+173>: movl   0x20(%rsp), %ebx
    0x7ffbd2e9ca91 <+177>: movl   %ebx, 0x30(%rsp)

(lldb) bt
* thread #1, stop reason = Exception 0x80000003 encountered at address 0x7ffbd2e9ca82
  * frame #0: 0x00007ffbd2e9ca83 ntdll.dll`RtlIsZeroMemory + 163
    frame #1: 0x00007ffbd2ea5b5a ntdll.dll`__misaligned_access + 1066
    frame #2: 0x00007ffbd2ea5e3a ntdll.dll`__misaligned_access + 1802
    frame #3: 0x00007ffbd2eb1e35 ntdll.dll`__misaligned_access + 50949
    frame #4: 0x00007ffbd2dcc3bc ntdll.dll`RtlGetCurrentServiceSessionId + 4892
    frame #5: 0x00007ffbd2dcb001 ntdll.dll`RtlFreeHeap + 81
    frame #6: 0x00007ffbd016364b ucrtbase.dll`_free_base + 27
    frame #7: 0x00007ff65dd213f0 a.exe`main + 96
    frame #8: 0x00007ff65dd21303 a.exe`__tmainCRTStartup at crtexe.c:259:15
    frame #9: 0x00007ff65dd21366 a.exe`mainCRTStartup at crtexe.c:180:9
    frame #10: 0x00007ffbd134259d kernel32.dll`BaseThreadInitThunk + 29
    frame #11: 0x00007ffbd2deaf38 ntdll.dll`RtlUserThreadStart + 40

thus I assume I am linking something incorrectly and some allocations made with ucrt are free with snmalloc or conversely which then makes ucrt scream.

[mjp41]

Using the static library will only override the allocations in the exe, but I think you are picking up a DLL as well, which won't have the allocations overridden.

There are some horrible things other allocators do to achieve this by rewriting things inside UCRT. But I have never gotten around to this in snmalloc.

[mjp41]

Just going to add a comment of the various approaches I have found.

• There is minhooks
  • with an example for using it with an allocator here:
    https://gist.github.com/Erfan-Ahmadi/93369b8b7fbce77b1adf8e244ea33849
  • and an explanation of how it works here: https://www.codeproject.com/KB/winsdk/LibMinHook.aspx
• Micro-malloc has an open source override for Windows in here: https://github.com/Thermadiag/micromalloc/tree/main/micro_proxy
• tcmalloc the older version patches various functions here: https://github.com/gperftools/gperftools/blob/master/src/windows/patch_functions.cc

I don't know as I have much time to follow up on this at the moment, but adding this comment in case anyone else does. Otherwise, it is useful for me when I get more time to work on this.

[jcelerier]

I see, I thought that since mimalloc did it (https://microsoft.github.io/mimalloc/overrides.html) likely snmalloc too but I understand now that this was a bold assumption :)

[jcelerier]

That said, what I find weird is that in the call trace I see:

frame #6: 0x00007ffbd016364b ucrtbase.dll`_free_base + 27
frame #7: 0x00007ff65dd213f0 a.exe`main + 96

(I since did a -g -O0 build too and could confirm that it goes straight from my main to _free_base). I'd expect to go through snmalloc's operator delete in that case, no ?

[mjp41]

I just looked at the static library by default creates the symbols

  sn_malloc, sn_free, ...

You can override the prefix by passing to CMake:

-DSNMALLOC_STATIC_LIBRARY_PREFIX=""

This would create malloc, free, ..., etc but I have sometimes had problems with this on Windows depending on other build configurations.

If you don't want to override free/malloc/etc just the C++ new/delete, then you can link libsnmalloc-new-override.a instead. This I have had better success with on projects on Windows.

https://github.com/microsoft/snmalloc/blob/d6ad197d3c8452cd4d2548813e0b88a333ce11d6/CMakeLists.txt#L486C12-L486C33

I am going to take a bit of time to investigate building a proper override using Detours, which would work across multiple DLLs.

[SchrodingerZhu]

Also, windows has a _set_new_hander hook which we does not yet override.

[Andarwinux]

Is it possible to build a snmalloc dll that exports symbols prefixed with "mi" and disguise it as "mimalloc-override.dll" for use with minject(mimalloc-redirect.dll)?

[mjp41]

Jean-Michaël Celerier (@jcelerier) I got around to implementing the Windows heap override using Detours. If you are interested in trying it, then take a look at

#775

This PR is, I believe, functional but very lightly tested at the moment.

[akrieger]

The detours implementation works for us in a purely statically linked binary, although given that scenario I'm inclined to just play whackamole with the private crt allocator symbols to avoid the overhead of the trampoline and extra detours dependency. We can just insert the extra needed symbols into our 'header only' implementation file for integrating snmalloc. Edit: nope, trying to shim _recalloc_base ended up causing reentrancy and a crash. I did find https://developercommunity.visualstudio.com/t/-msize-implementation/626773 about the _msize_base issue but sadly no movement on that in... a long time.