92 commits

Author	SHA1	Message	Date
daan	b77be05e40	only collect retired at fresh page allocation	2020-01-20 12:14:34 -08:00
daan	dc58388968	Add ability to register custom error function called on various error conditions; including ENOMEM	2020-01-17 19:59:55 -08:00
daan	c6037ac8f2	fix assertion index	2020-01-16 16:56:02 -08:00
daan	ad32eb1dfb	eager collect on page reclamation	2020-01-15 17:57:19 -08:00
daan	0099707af9	use delayed free for all pages; reduce size of the page structure for improved address calculation	2020-01-15 17:19:01 -08:00
daan	0956a05bf6	add argument pointer to the register deferred free callback	2020-01-15 10:21:32 -08:00
daan	12ef2816ed	fix bug exposed by commit 59fa286 where reclaimed pages could be stuck to NEVER_DELAYED	2020-01-08 19:00:03 -08:00
daan	45582d1fb5	revert a2a9230 (remove empty page removal on search): this is not generally valid when concurrent frees do not always add to thread_delayed_free.	2020-01-05 13:58:49 -08:00
daan	59fa286294	fix bug where continue would wrongly exit the do-while loop for delayed freeing	2020-01-05 13:52:07 -08:00
daan	a2a9230ad6	remove empty page removal on page search (no longer needed with retired collection and delayed freeing)	2020-01-03 22:52:52 -08:00
daan	2b108c8748	increase retire expiration to 4	2020-01-03 21:39:18 -08:00
daan	eeb623e6af	increase retire limit, collect retired pages	2020-01-03 17:06:41 -08:00
daan	77134e1ad0	update free list encoding to stronger formula with addition last	2019-12-28 15:17:49 -08:00
daan	e3391d9a53	stronger encoding of free lists using two keys per page	2019-12-28 00:57:42 -08:00
daan	ba87a39d9f	updated random cookie generation using OS primitives and chacha20	2019-12-22 19:37:49 -08:00
daan	c6df7a199c	experimental eager page commit option	2019-11-24 22:02:40 -08:00
daan	6568059cc6	merge from dev	2019-11-21 16:28:28 -08:00
daan	74dbfc30be	improved security by encoding NULL values; double free mitigation on by default; more precise free list corruption detection	2019-11-21 15:21:23 -08:00
Daan Leijen	211f1aa519	remove reset delay slots; add reset tracking per page and segment	2019-11-20 14:55:12 -08:00
Daan Leijen	56b9fac4bf	merge from dev	2019-11-20 13:31:04 -08:00
Daan Leijen	4d4a2885f5	use atomic read/write on the page->heap field where concurrent interaction is possible	2019-11-20 13:19:17 -08:00
daan	6ccfb5a93e	merge from dev	2019-11-07 11:00:50 -08:00
daan	9b65388807	fix space leak in secure mode where a non-null free list would be discarded	2019-11-07 10:59:19 -08:00
daan	36090dee5c	Merge branch 'dev' into dev-exp	2019-11-07 10:34:06 -08:00
daan	31d11f64d5	fix secure free list extension where a non-empty initial free list was discarded	2019-11-07 10:33:45 -08:00
daan	378716c467	refactor and improve atomic bitmap usage	2019-11-07 10:26:52 -08:00
daan	9d6a5acb22	fix unix build warnings	2019-11-03 13:34:54 -08:00
daan	8725a88fba	fix assertion in debug secure mode	2019-10-30 15:22:40 -07:00
daan	4a4d74927c	protect against double-free in multi-threaded free list	2019-10-30 14:53:21 -07:00
daan	b052d3b731	enable double free and heap corruption detection in debug mode	2019-10-28 15:54:33 -07:00
daan	87bdfbb9b6	use more conservative retire strategy	2019-10-28 14:12:06 -07:00
daan	081e2d1eb6	fix statistics display	2019-10-28 13:43:42 -07:00
daan	28d4ec4c5a	fix statistics accounting of huge pages	2019-10-28 13:14:14 -07:00
daan	5de851a84d	update page_flags to have more portable definition	2019-10-17 16:48:16 -07:00
daan	60efb62155	make secure a build option only	2019-09-17 17:31:42 -07:00
Daan Leijen	1909cfb346	refine mi_os_good_alloc_size and use it for huge pages (to ensure realloc is bounded as in #153	2019-09-10 13:26:51 -07:00
Daan Leijen	d278c26c0e	round huge page sizes into 12.5% increments (issue #153)	2019-09-10 07:54:31 -07:00
daan	b104e434e4	fix volatile declaration on function	2019-09-09 08:07:35 -07:00
daan	23155c5d71	add recursion guard to deferred callback	2019-09-09 08:02:41 -07:00
daan	89858ab791	improve page_free_extend performance slightly; do not maintain expensive statistics in release mode	2019-09-06 16:48:48 -07:00
daan	7a91368108	pass page to block_zero_init and extend to block_size	2019-09-03 10:11:24 -07:00
daan	739d11313c	fix commit zero on windows	2019-09-02 14:53:22 -07:00
daan	d1bd1644d5	support zero-initialized memory detection	2019-09-02 13:16:52 -07:00
daan	64c1d6de86	fix mi_likely branch that was marked as unlikely	2019-08-29 07:50:35 -07:00
daan	a654732d99	merge from dev new atomics	2019-08-26 12:30:03 -07:00
daan	e8664001f7	Use standard _Atomic declarations and clean up atomic operations	2019-08-25 22:59:12 -07:00
daan	23c35f4aba	merge from dev	2019-08-25 13:07:07 -07:00
daan	038e8fd7d6	reduce retire size	2019-08-25 12:10:18 -07:00
daan	a431d80fc3	better object sizes for large objects	2019-08-25 10:20:03 -07:00
daan	23812cc0ac	do not keep a queue of huge pages and free them directly	2019-08-24 15:45:14 -07:00