Merge branch 'dev' into dev

This commit is contained in:
Daan 2024-12-30 12:27:46 -08:00 committed by GitHub
commit f33aa58d88
WARNING! Although there is a key with this ID in the database it does not verify this commit! This commit is SUSPICIOUS.
GPG key ID: B5690EEEBB952194
79 changed files with 3509 additions and 3833 deletions

View file

@ -164,7 +164,7 @@ The `mimalloc` project builds a static library (in `out/msvc-x64`), while the
`mimalloc-override` project builds a DLL for overriding malloc
in the entire program.
## macOS, Linux, BSD, etc.
## Linux, macOS, BSD, etc.
We use [`cmake`](https://cmake.org)<sup>1</sup> as the build system:
@ -200,13 +200,26 @@ free lists, etc., as:
> make
```
This will name the shared library as `libmimalloc-secure.so`.
Use `ccmake`<sup>2</sup> instead of `cmake`
to see and customize all the available build options.
Use `cmake ../.. -LH` to see all the available build options.
Notes:
1. Install CMake: `sudo apt-get install cmake`
2. Install CCMake: `sudo apt-get install cmake-curses-gui`
The examples use the default compiler. If you like to use another, use:
```
> CC=clang CXX=clang++ cmake ../..
```
## Cmake with Visual Studio
You can also use cmake on Windows. Open a Visual Studio development prompt
and invoke `cmake` with the right [generator](https://cmake.org/cmake/help/latest/generator/Visual%20Studio%2017%202022.html)
and architecture, like:
```
> cmake ..\.. -G "Visual Studio 17 2022" -A x64 -DMI_OVERRIDE=ON
```
The cmake build type is specified when actually building, for example:
```
> cmake --build . --config=Release
```
## Single source
@ -240,7 +253,7 @@ to link with the static library. See `test\CMakeLists.txt` for an example.
For best performance in C++ programs, it is also recommended to override the
global `new` and `delete` operators. For convenience, mimalloc provides
[`mimalloc-new-delete.h`](https://github.com/microsoft/mimalloc/blob/master/include/mimalloc-new-delete.h) which does this for you -- just include it in a single(!) source file in your project.
[`mimalloc-new-delete.h`](include/mimalloc-new-delete.h) which does this for you -- just include it in a single(!) source file in your project.
In C++, mimalloc also provides the `mi_stl_allocator` struct which implements the `std::allocator`
interface.
@ -415,27 +428,33 @@ Note that certain security restrictions may apply when doing this from
the [shell](https://stackoverflow.com/questions/43941322/dyld-insert-libraries-ignored-when-calling-application-through-bash).
### Dynamic Override on Windows
# Windows Override
<span id="override_on_windows">Dynamically overriding on mimalloc on Windows</span>
is robust and has the particular advantage to be able to redirect all malloc/free calls that go through
the (dynamic) C runtime allocator, including those from other DLL's or libraries.
As it intercepts all allocation calls on a low level, it can be used reliably
<span id="override_on_windows">Dynamically overriding on mimalloc on Windows</span>
is robust and has the particular advantage to be able to redirect all malloc/free calls
that go through the (dynamic) C runtime allocator, including those from other DLL's or
libraries. As it intercepts all allocation calls on a low level, it can be used reliably
on large programs that include other 3rd party components.
There are four requirements to make the overriding work robustly:
There are four requirements to make the overriding work well:
1. Use the C-runtime library as a DLL (using the `/MD` or `/MDd` switch).
2. Link your program explicitly with `mimalloc-override.dll` library.
To ensure the `mimalloc-override.dll` is loaded at run-time it is easiest to insert some
call to the mimalloc API in the `main` function, like `mi_version()`
(or use the `/INCLUDE:mi_version` switch on the linker). See the `mimalloc-override-test` project
for an example on how to use this.
3. The [`mimalloc-redirect.dll`](bin) (or `mimalloc-redirect32.dll`) must be put
in the same folder as the main `mimalloc-override.dll` at runtime (as it is a dependency of that DLL).
The redirection DLL ensures that all calls to the C runtime malloc API get redirected to
mimalloc functions (which reside in `mimalloc-override.dll`).
4. Ensure the `mimalloc-override.dll` comes as early as possible in the import
2. Link your program explicitly with the `mimalloc.lib` export library for the `mimalloc.dll`.
(which must be compiled with `-DMI_OVERRIDE=ON`, which is the default though).
To ensure the `mimalloc.dll` is actually loaded at run-time it is easiest
to insert some call to the mimalloc API in the `main` function, like `mi_version()`
(or use the `/include:mi_version` switch on the linker command, or
similarly, `#pragma comment(linker, "/include:mi_version")` in some source file).
See the `mimalloc-test-override` project for an example on how to use this.
3. The `mimalloc-redirect.dll` must be put in the same folder as the main
`mimalloc.dll` at runtime (as it is a dependency of that DLL).
The redirection DLL ensures that all calls to the C runtime malloc API get
redirected to mimalloc functions (which reside in `mimalloc.dll`).
4. Ensure the `mimalloc.dll` comes as early as possible in the import
list of the final executable (so it can intercept all potential allocations).
You can use `minject -l <exe>` to check this if needed.
For best performance on Windows with C++, it
is also recommended to also override the `new`/`delete` operations (by including
@ -443,15 +462,14 @@ is also recommended to also override the `new`/`delete` operations (by including
a single(!) source file in your project).
The environment variable `MIMALLOC_DISABLE_REDIRECT=1` can be used to disable dynamic
overriding at run-time. Use `MIMALLOC_VERBOSE=1` to check if mimalloc was successfully redirected.
overriding at run-time. Use `MIMALLOC_VERBOSE=1` to check if mimalloc was successfully
redirected.
For different platforms than x64, you may need a specific [redirection dll](bin).
Furthermore, we cannot always re-link an executable or ensure `mimalloc.dll` comes
first in the import table. In such cases the [`minject`](bin) tool can be used
to patch the executable's import tables.
We cannot always re-link an executable with `mimalloc-override.dll`, and similarly, we cannot always
ensure the the DLL comes first in the import table of the final executable.
In many cases though we can patch existing executables without any recompilation
if they are linked with the dynamic C runtime (`ucrtbase.dll`) -- just put the `mimalloc-override.dll`
into the import table (and put `mimalloc-redirect.dll` in the same folder)
Such patching can be done for example with [CFF Explorer](https://ntcore.com/?page_id=388) or
the [`minject`](bin) program.
## Static override
@ -469,7 +487,7 @@ object file. For example:
Another way to override statically that works on all platforms, is to
link statically to mimalloc (as shown in the introduction) and include a
header file in each source file that re-defines `malloc` etc. to `mi_malloc`.
This is provided by [`mimalloc-override.h`](https://github.com/microsoft/mimalloc/blob/master/include/mimalloc-override.h). This only works reliably though if all sources are
This is provided by [`mimalloc-override.h`](include/mimalloc-override.h). This only works reliably though if all sources are
under your control or otherwise mixing of pointers from different heaps may occur!
@ -505,9 +523,13 @@ you also need to tell `valgrind` to not intercept those calls itself, and use:
By setting the `MIMALLOC_SHOW_STATS` environment variable you can check that mimalloc is indeed
used and not the standard allocator. Even though the [Valgrind option][valgrind-soname]
is called `--soname-synonyms`, this also
works when overriding with a static library or object file. Unfortunately, it is not possible to
dynamically override mimalloc using `LD_PRELOAD` together with `valgrind`.
is called `--soname-synonyms`, this also works when overriding with a static library or object file.
To dynamically override mimalloc using `LD_PRELOAD` together with `valgrind`, use:
```
> valgrind --trace-children=yes --soname-synonyms=somalloc=*mimalloc* /usr/bin/env LD_PRELOAD=/usr/lib/libmimalloc.so -- <myprogram>
```
See also the `test/test-wrong.c` file to test with `valgrind`.
Valgrind support is in its initial development -- please report any issues.