400 commits

Author	SHA1	Message	Date
Sergiy Kuryata	d19f103ea3	Handle large allocations in separate segments	2024-12-05 19:44:52 -08:00
Sergiy Kuryata	a9bd3205be	Try separating pages with different slice counts	2024-12-04 23:21:43 -08:00
Sergiy Kuryata	8dd6460d5e	Do not discard segments for large object allocations	2024-11-30 18:44:35 -08:00
Sergiy Kuryata	0be44b2b0f	Add global counters for segments and allocation stats	2024-11-26 16:32:54 -08:00
Sergiy Kuryata	75eec9e61d	Fix free space mask accounting for unused slices	2024-11-19 18:41:30 -08:00
Sergiy Kuryata	02f3e0fe7a	Switch to 64bit bitmask and bug fixes	2024-11-08 17:35:14 -08:00
Sergiy Kuryata	911de68d3e	Experimental optimization for the segment reclamation logic. ... The goal is to improve efficiency of finding an abandoned segment that contains free space of the required size. This change adds a free space bitmask to the segment object. More optimal place for the bitmask could be in the arena (i.e. extend the arena’s bitmask for abandoned segments to include size information too).	2024-10-31 16:53:14 -07:00
Sergiy Kuryata	b604099334	Experimental improvements to reduce memory usage ... 1. Prefer to use more loaded pages when allocating 2. Add an option to restrict unbounded heap growth and allow the excessive memory allocations to be re-used by other threads	2024-10-11 10:35:51 -07:00
Daan	65a0c82ac9	merge from dev (support for heap tag, upstream of python/cpython#113742	2024-05-19 08:10:35 -07:00
Daan	0c4041fa53	add support for custom heap and page tags, upstream of python/cpython#113742	2024-05-19 08:06:32 -07:00
Daan	4a26a4568e	fix out-of-bounds write on span free in huge segments	2024-05-16 14:26:05 -07:00
Daan	44b65b19df	remove pre_size parameter for slices	2024-05-16 13:30:33 -07:00
Daan	e8206e9f6f	merge from dev improved aligned allocation	2024-05-11 07:08:48 -07:00
Daan	7128db7bba	simplified aligned allocation; improved codegen; fix mi_good_size with padding included; add MI_MAX_ALIGN_GUARANTEE	2024-05-11 06:43:52 -07:00
Daan	83af243bca	merge from dev	2024-04-19 12:50:32 -07:00
Daan	09e91ec911	add assert for segment purges	2024-04-19 12:44:23 -07:00
Daan	bf5932c3c6	use better purging for segments_collect	2024-04-19 12:43:05 -07:00
Daan	2b7530e183	add segment_collect for forced heap_collect	2024-04-19 12:33:17 -07:00
Daan	35658681e9	purge segments on forced collect (issue #878)	2024-04-19 12:29:49 -07:00
Daan	32e065bb32	rename segment_thread_collect to segment_collect	2024-04-19 12:11:58 -07:00
Daan	78418b3d24	fix overflow of slice count, PR #877	2024-04-19 10:03:14 -07:00
daanx	630521e8db	merge from dev	2024-03-25 16:05:39 -07:00
Daan Leijen	460278f110	comments	2024-03-25 16:02:20 -07:00
Daan Leijen	c1d7d7f563	merge from dev	2024-03-25 15:32:56 -07:00
Daan Leijen	0022802177	commend and make at least 8 tries for reclaim	2024-03-25 15:25:04 -07:00
Daan Leijen	cc8d89a085	update comments	2024-03-25 07:35:49 -07:00
Daan Leijen	af3f2f9168	fix assertions and warnings on gcc	2024-03-24 22:32:32 -07:00
Daan Leijen	3c85983a35	merge from dev with new page fields (block_size and is_huge)	2024-03-24 22:12:33 -07:00
daanx	006ae2d055	add is_huge page flag to ensure the right page queue is returned (see #868)	2024-03-24 17:07:28 -07:00
Daan Leijen	b5665f0eec	add full block_size and page_start to page info	2024-03-24 14:17:17 -07:00
Daan Leijen	6688b45fbd	rename MI_ALIGNMENT_MAX to MI_BLOCK_ALIGNMENT_MAX for clarity	2024-03-24 10:57:02 -07:00
Daan Leijen	f141ca12a4	add extra runtime check to ensure we never insert large or huge pages in the segment free queue (issue #870)	2024-03-24 10:53:39 -07:00
Daan Leijen	6399dbdc30	merge from dev (new free.c)	2024-03-24 09:56:26 -07:00
Daan Leijen	4f809aadb7	use free field for expiration instead of used	2024-03-24 08:29:56 -07:00
Daan	2cca58dfc8	Merge branch 'dev' into dev-slice	2024-03-03 18:08:27 -08:00
Daan	da52048025	rename local to avoid warning in static build	2024-03-03 18:08:17 -08:00
daanx	abb8eab9b2	merge from dev	2024-03-03 17:43:52 -08:00
daanx	5fe83bf327	limit reclaim from free to half the segments to prevent a pure freeing thread to reclaim too many segments	2024-03-03 17:38:50 -08:00
daanx	5a2ed6d977	fix assertion	2024-03-02 18:27:06 -08:00
daanx	8e1f8f4d5c	Merge branch 'dev-abandon' into dev-slice-abandon	2024-03-02 18:25:09 -08:00
daanx	43d40704b9	keep threadid 0 for abandoned_next to preserve invariants	2024-03-02 18:24:53 -08:00
daanx	3090f23c25	fix return value of get_reclaim_tries	2024-03-01 18:59:13 -08:00
daanx	2c433a2b23	merge from dev-abandon	2024-03-01 18:54:29 -08:00
daanx	500d2ad1fd	ensure non-arena segments are force purged on abandonment; make non-arena reclaim atomic	2024-03-01 18:47:59 -08:00
daanx	280123bd5c	purge on page free as well	2024-03-01 15:25:57 -08:00
daanx	f6320bd3be	merge from dev-abandon	2024-03-01 15:15:24 -08:00
daanx	c4f1f2e079	make reclaim tries a percentage of the abandoned count	2024-03-01 14:57:46 -08:00
daanx	e5d1cb3092	merge from dev-abandon	2024-03-01 10:56:43 -08:00
daanx	cf8f73098e	start abandoned search randomized	2024-03-01 10:51:18 -08:00
daanx	71bcf1c76b	maintain abandoned_count more robustly	2024-03-01 10:31:58 -08:00