kiper220/mimalloc
1
0
Fork 0
mirror of https://github.com/microsoft/mimalloc.git synced 2025-05-02 13:49:31 +03:00
Code Issues Projects Releases Packages Wiki Activity Actions

fix large OS page behaviour on Linux; default is now 2 which only uses large OS pages (not huge) through madvise

Browse source
This commit is contained in:
Daan Leijen 2025-01-03 20:00:40 -08:00
parent 27687718bc
commit 2451b5685c
4 changed files with 16 additions and 12 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
doc/mimalloc-doc.h
View file

@ -1231,8 +1231,9 @@ Further options for large workloads and services:
the actual NUMA nodes is fine and will only cause threads to potentially allocate more memory across actual NUMA
nodes (but this can happen in any case as NUMA local allocation is always a best effort but not guaranteed).
- `MIMALLOC_ALLOW_LARGE_OS_PAGES=1`: use large OS pages (2 or 4MiB) when available; for some workloads this can significantly
improve performance. When this option is disabled, it also disables transparent huge pages (THP) for the process
(on Linux and Android). Use `MIMALLOC_VERBOSE` to check if the large OS pages are enabled -- usually one needs
improve performance. When this option is disabled (default), it also disables transparent huge pages (THP) for the process
(on Linux and Android). On Linux the default setting is 2 -- this enables the use of large pages through THP only.
Use `MIMALLOC_VERBOSE` to check if the large OS pages are enabled -- usually one needs
to explicitly give permissions for large OS pages (as on [Windows][windows-huge] and [Linux][linux-huge]). However, sometimes
the OS is very slow to reserve contiguous physical memory for large OS pages so use with care on systems that
can have fragmented memory (for that reason, we generally recommend to use `MIMALLOC_RESERVE_HUGE_OS_PAGES` instead whenever possible).

13
readme.md
View file

@ -12,8 +12,8 @@ is a general purpose allocator with excellent [performance](#performance) charac
Initially developed by Daan Leijen for the runtime systems of the
[Koka](https://koka-lang.github.io) and [Lean](https://github.com/leanprover/lean) languages.
Latest release tag: `v2.1.8` (2025-01-03).
Latest v1 tag: `v1.8.8` (2024-01-03).
Latest release tag: `v2.1.9` (2025-01-03).
Latest v1 tag: `v1.8.9` (2024-01-03).
mimalloc is a drop-in replacement for `malloc` and can be used in other programs
without code changes, for example, on dynamically linked ELF-based systems (Linux, BSD, etc.) you can use it as:
@ -81,7 +81,7 @@ Enjoy!
### Releases
* 2025-01-03, `v1.8.8`, `v2.1.8`, `v3.0-alpha`: Interim release. Support Windows arm64. New [guarded](#guarded) build that can place OS
* 2025-01-03, `v1.8.9`, `v2.1.9`, `v3.0-alpha`: Interim release. Support Windows arm64. New [guarded](#guarded) build that can place OS
guard pages behind objects to catch buffer overflows as they occur.
Many small fixes: build on Windows arm64, cygwin, riscV, and dragonfly; fix Windows static library initialization to account for
thread local destructors (in Rust/C++); macOS tag change; macOS TLS slot fix; improve stats;
@ -342,9 +342,10 @@ Further options for large workloads and services:
at runtime. Setting `N` to 1 may avoid problems in some virtual environments. Also, setting it to a lower number than
the actual NUMA nodes is fine and will only cause threads to potentially allocate more memory across actual NUMA
nodes (but this can happen in any case as NUMA local allocation is always a best effort but not guaranteed).
- `MIMALLOC_ALLOW_LARGE_OS_PAGES=1`: use large OS pages (2 or 4MiB) when available; for some workloads this can significantly
improve performance. When this option is disabled, it also disables transparent huge pages (THP) for the process
(on Linux and Android). Use `MIMALLOC_VERBOSE` to check if the large OS pages are enabled -- usually one needs
- `MIMALLOC_ALLOW_LARGE_OS_PAGES=0`: Set to 1 to use large OS pages (2 or 4MiB) when available; for some workloads this can significantly
improve performance. When this option is disabled (default), it also disables transparent huge pages (THP) for the process
(on Linux and Android). On Linux the default setting is 2 -- this enables the use of large pages through THP only.
Use `MIMALLOC_VERBOSE` to check if the large OS pages are enabled -- usually one needs
to explicitly give permissions for large OS pages (as on [Windows][windows-huge] and [Linux][linux-huge]). However, sometimes
the OS is very slow to reserve contiguous physical memory for large OS pages so use with care on systems that
can have fragmented memory (for that reason, we generally recommend to use `MIMALLOC_RESERVE_HUGE_OS_PAGES` instead whenever possible).

2
src/options.c
View file

@ -80,7 +80,7 @@ typedef struct mi_option_desc_s {
#ifndef MI_DEFAULT_ALLOW_LARGE_OS_PAGES
#if defined(__linux__) && !defined(__ANDROID__)
#define MI_DEFAULT_ALLOW_LARGE_OS_PAGES 1
#define MI_DEFAULT_ALLOW_LARGE_OS_PAGES 2 // enabled, but only use transparent huge pages through madvise
#else
#define MI_DEFAULT_ALLOW_LARGE_OS_PAGES 0
#endif

8
src/prim/unix/prim.c
View file

@ -272,7 +272,7 @@ static void* unix_mmap(void* addr, size_t size, size_t try_alignment, int protec
protect_flags |= PROT_MAX(PROT_READ | PROT_WRITE); // BSD
#endif
// huge page allocation
if ((large_only || _mi_os_use_large_page(size, try_alignment)) && allow_large) {
if (allow_large && (large_only || (_mi_os_use_large_page(size, try_alignment) && mi_option_get(mi_option_allow_large_os_pages) == 1))) {
static _Atomic(size_t) large_page_try_ok; // = 0;
size_t try_ok = mi_atomic_load_acquire(&large_page_try_ok);
if (!large_only && try_ok > 0) {
@ -293,7 +293,7 @@ static void* unix_mmap(void* addr, size_t size, size_t try_alignment, int protec
#endif
#ifdef MAP_HUGE_1GB
static bool mi_huge_pages_available = true;
if ((size % MI_GiB) == 0 && mi_huge_pages_available) {
if (large_only && (size % MI_GiB) == 0 && mi_huge_pages_available) {
lflags |= MAP_HUGE_1GB;
}
else
@ -313,7 +313,9 @@ static void* unix_mmap(void* addr, size_t size, size_t try_alignment, int protec
#ifdef MAP_HUGE_1GB
if (p == NULL && (lflags & MAP_HUGE_1GB) == MAP_HUGE_1GB) {
mi_huge_pages_available = false; // don't try huge 1GiB pages again
_mi_warning_message("unable to allocate huge (1GiB) page, trying large (2MiB) pages instead (errno: %i)\n", errno);
if (large_only) {
_mi_warning_message("unable to allocate huge (1GiB) page, trying large (2MiB) pages instead (errno: %i)\n", errno);
}
lflags = ((lflags & ~MAP_HUGE_1GB) | MAP_HUGE_2MB);
p = unix_mmap_prim(addr, size, try_alignment, protect_flags, lflags, lfd);
}