aboutsummaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)Author
2015-08-10Revert to first-best-fit run/chunk allocation.android-m-preview-2Jason Evans
This effectively reverts 97c04a93838c4001688fe31bf018972b4696efe2 (Use first-fit rather than first-best-fit run/chunk allocation.). In some pathological cases, first-fit search dominates allocation time, and it also tends not to converge as readily on a steady state of memory layout, since precise allocation order has a bigger effect than for first-best-fit. (cherry picked from upstream commit https://github.com/jemalloc/jemalloc/commit/aa2826621e1793db9faea31e803690ccbe36f14c) Bug: 22172059 (cherry picked from commit 486fa41f4ace88a9d1b96831bc36cbd2cfa4957e) Change-Id: I88f7b5bc3b18c953cc9ad8ce4a54c5a0c582c613
2015-08-10On 32 bit systems, force huge allocs on arena 0.Christopher Ferris
The new version of jemalloc has moved all huge allocations from a single cache, to a per arena cache. This can result in virtual address space exhaustion when someone has a pattern where they allocate huge allocations from new threads all of the time. Bug: 22172059 (cherry picked from commit dcfc29b2aa1ee53fe4b40368d02f80ca4bddeabc) Change-Id: I0f380c1295a49eb76390cf5f3c9285e685122706
2015-08-07Implement chunk hook support for page run commit/decommit.Jason Evans
Cascade from decommit to purge when purging unused dirty pages, so that it is possible to decommit cleaned memory rather than just purging. For non-Windows debug builds, decommit runs rather than purging them, since this causes access of deallocated runs to segfault. This resolves #251.
2015-08-06Fix an in-place growing large reallocation regression.Jason Evans
Fix arena_ralloc_large_grow() to properly account for large_pad, so that in-place large reallocation succeeds when possible, rather than always failing. This regression was introduced by 8a03cf039cd06f9fa6972711195055d865673966 (Implement cache index randomization for large allocations.)
2015-08-04MSVC compatibility changesMatthijs
- Decorate public function with __declspec(allocator) and __declspec(restrict), just like MSVC 1900 - Support JEMALLOC_HAS_RESTRICT by defining the restrict keyword - Move __declspec(nothrow) between 'void' and '*' so it compiles once more
2015-08-03Generalize chunk management hooks.Jason Evans
Add the "arena.<i>.chunk_hooks" mallctl, which replaces and expands on the "arena.<i>.chunk.{alloc,dalloc,purge}" mallctls. The chunk hooks allow control over chunk allocation/deallocation, decommit/commit, purging, and splitting/merging, such that the application can rely on jemalloc's internal chunk caching and retaining functionality, yet implement a variety of chunk management mechanisms and policies. Merge the chunks_[sz]ad_{mmap,dss} red-black trees into chunks_[sz]ad_retained. This slightly reduces how hard jemalloc tries to honor the dss precedence setting; prior to this change the precedence setting was also consulted when recycling chunks. Fix chunk purging. Don't purge chunks in arena_purge_stashed(); instead deallocate them in arena_unstash_purged(), so that the dirty memory linkage remains valid until after the last time it is used. This resolves #176 and #201.
2015-07-24Implement support for non-coalescing maps on MinGW.Jason Evans
- Do not reallocate huge objects in place if the number of backing chunks would change. - Do not cache multi-chunk mappings. This resolves #213.
2015-07-24Fix huge_ralloc_no_move() to succeed more often.Jason Evans
Fix huge_ralloc_no_move() to succeed if an allocation request results in the same usable size as the existing allocation, even if the request size is smaller than the usable size. This bug did not cause correctness issues, but it could cause unnecessary moves during reallocation.
2015-07-23Fix huge_palloc() to handle size rather than usize input.Jason Evans
huge_ralloc() passes a size that may not be precisely a size class, so make huge_palloc() handle the more general case of a size input rather than usize. This regression appears to have been introduced by the addition of in-place huge reallocation; as such it was never incorporated into a release.
2015-07-23Change arena_palloc_large() parameter from size to usize.Jason Evans
This change merely documents that arena_palloc_large() always receives usize as its argument.
2015-07-23Fix MinGW-related portability issues.Jason Evans
Create and use FMT* macros that are equivalent to the PRI* macros that inttypes.h defines. This allows uniform use of the Unix-specific format specifiers, e.g. "%zu", as well as avoiding Windows-specific definitions of e.g. PRIu64. Add ffs()/ffsl() support for compiling with gcc. Extract compatibility definitions of ENOENT, EINVAL, EAGAIN, EPERM, ENOMEM, and ENORANGE into include/msvc_compat/windows_extra.h and use the file for tests as well as for core jemalloc code.
2015-07-22Add JEMALLOC_FORMAT_PRINTF().Jason Evans
Replace JEMALLOC_ATTR(format(printf, ...). with JEMALLOC_FORMAT_PRINTF(), so that configuration feature tests can omit the attribute if it would cause extraneous compilation warnings.
2015-07-21Move JEMALLOC_NOTHROW just after return type.Jason Evans
Only use __declspec(nothrow) in C++ mode. This resolves #244.
2015-07-21Remove JEMALLOC_ALLOC_SIZE annotations on functions not returning pointersMike Hommey
As per gcc documentation: The alloc_size attribute is used to tell the compiler that the function return value points to memory (...) This resolves #245.
2015-07-17Add the config.cache_oblivious mallctl.Jason Evans
2015-07-15Revert to first-best-fit run/chunk allocation.Jason Evans
This effectively reverts 97c04a93838c4001688fe31bf018972b4696efe2 (Use first-fit rather than first-best-fit run/chunk allocation.). In some pathological cases, first-fit search dominates allocation time, and it also tends not to converge as readily on a steady state of memory layout, since precise allocation order has a bigger effect than for first-best-fit.
2015-07-10Avoid function prototype incompatibilities.Jason Evans
Add various function attributes to the exported functions to give the compiler more information to work with during optimization, and also specify throw() when compiling with C++ on Linux, in order to adequately match what __THROW does in glibc. This resolves #237.
2015-07-10Revert "Fix chunk size check."Christopher Ferris
Bug: 21633176 This reverts commit c429191e01e516d97bd501465289a351ef48d63a. Change-Id: Idbc8dee0e5017c63654b38839bad7cc69619de56
2015-07-09Make sure the arena cache cannot be reused.Christopher Ferris
The arena cache is freed when the jemalloc key is destroyed. However, if another key destroy function winds up making an allocation, jemalloc can wind up rewriting the arena pointers into the previous arena cache pointer. The fix is to clear out the pointer and make sure that the arena cache will not be recreated. Bug: 20296122 (cherry picked from commit 10e814ba3f01361999d824f0c67db179580d4ca2) Change-Id: Ib1bb15144eba369df36e81e967fbc9c65f94aa4e
2015-07-07Fix a variable declaration typo.Jason Evans
2015-07-07Use jemalloc_ffs() rather than ffs().Jason Evans
2015-07-07Fix MinGW build warnings.Jason Evans
Conditionally define ENOENT, EINVAL, etc. (was unconditional). Add/use PRIzu, PRIzd, and PRIzx for use in malloc_printf() calls. gcc issued (harmless) warnings since e.g. "%zu" should be "%Iu" on Windows, and the alternative to this workaround would have been to disable the function attributes which cause gcc to look for type mismatches in formatted printing function calls.
2015-07-07Fix an assignment type warning for tls_callback.Jason Evans
2015-07-07Move a variable declaration closer to its use.Jason Evans
2015-06-25Optimizations for WindowsMatthijs
- Set opt_lg_chunk based on run-time OS setting - Verify LG_PAGE is compatible with run-time OS setting - When targeting Windows Vista or newer, use SRWLOCK instead of CRITICAL_SECTION - When targeting Windows Vista or newer, statically initialize init_lock
2015-06-25Revert "Temporary patch to avoid crashes."Christopher Ferris
This reverts commit 165e51a7a9067485e7b54b14b571020d7d9bdfc5. Bug: 21873627 Change-Id: I3245a7e9cede9d2e571ae2f994c0defd4172e3e6
2015-06-23Fix size class overflow handling when profiling is enabled.Jason Evans
Fix size class overflow handling for malloc(), posix_memalign(), memalign(), calloc(), and realloc() when profiling is enabled. Remove an assertion that erroneously caused arena_sdalloc() to fail when profiling was enabled. This resolves #232.
2015-06-22Convert arena_maybe_purge() recursion to iteration.Jason Evans
This resolves #235.
2015-06-22Add alignment assertions to public aligned allocation functions.Jason Evans
2015-06-22Fix two valgrind integration regressions.Jason Evans
The regressions were never merged into the master branch.
2015-06-08Fix chunk size check.Christopher Ferris
If the chunk returned in chunk_recycle is too small to satisfy the current request, the code would still allow it to be reused. Make sure the chunk size is checked in all cases: Bug: 21633176 Change-Id: I04d2a682efaa3c177960b68a143afd2f6dbe17c4
2015-05-29Clarify relationship between stats.resident and stats.mapped.Jason Evans
2015-05-29Bypass tcache when draining quarantined allocations.Jason Evans
This avoids the potential surprise of deallocating an object with one tcache specified, and having the object cached in a different tcache once it drains from the quarantine.
2015-05-20Impose a minimum tcache count for small size classes.Jason Evans
Now that small allocation runs have fewer regions due to run metadata residing in chunk headers, an explicit minimum tcache count is needed to make sure that tcache adequately amortizes synchronization overhead. Bug: 21326736 (cherry picked from commit 836bbe9951a903b2d76af53dfb3ad53ad186f8b9) Change-Id: I4aaf29240799642de77b3b79a3507d2789f23542
2015-05-20Fix performance regression in arena_palloc().Jason Evans
Pass large allocation requests to arena_malloc() when possible. This regression was introduced by 155bfa7da18cab0d21d87aa2dce4554166836f5d (Normalize size classes.). Bug: 21326736 (cherry picked from commit 5154175cf1e6e7b1a2ed0295c232e60384944b3f) Change-Id: Ie72eb63fcc9e26722df6b81e95a3bc6ddfc5cd99
2015-05-20Fix nhbins calculation.Jason Evans
This regression was introduced by 155bfa7da18cab0d21d87aa2dce4554166836f5d (Normalize size classes.). Bug: 21326736 (cherry picked from commit 5aa50a2834fb09c5338f0e7b9db49cc0edd1a38a) Change-Id: I0e739c3b93ab352e802832df861da704ee921a6c
2015-05-20Avoid atomic operations for dependent rtree reads.Jason Evans
Bug: 21326736 (cherry picked from commit fd5f9e43c35b39740e218fececbb70d929546bb0) Change-Id: Ie3b3d15a7bd18cb48b05b71f2df2b238517f019c
2015-05-20Implement cache index randomization for large allocations.Jason Evans
Extract szad size quantization into {extent,run}_quantize(), and . quantize szad run sizes to the union of valid small region run sizes and large run sizes. Refactor iteration in arena_run_first_fit() to use run_quantize{,_first,_next(), and add support for padded large runs. For large allocations that have no specified alignment constraints, compute a pseudo-random offset from the beginning of the first backing page that is a multiple of the cache line size. Under typical configurations with 4-KiB pages and 64-byte cache lines this results in a uniform distribution among 64 page boundary offsets. Add the --disable-cache-oblivious option, primarily intended for performance testing. This resolves #13. Bug: 21326736 (cherry picked from commit 8a03cf039cd06f9fa6972711195055d865673966) Change-Id: I8150df74d4bc5bfc8b30dcb162fb9b087527d4c0
2015-05-20Rename pprof to jeprof.Jason Evans
This rename avoids installation collisions with the upstream gperftools. Additionally, jemalloc's per thread heap profile functionality introduced an incompatible file format, so it's now worthwhile to clearly distinguish jemalloc's version of this script from the upstream version. This resolves #229. Bug: 21326736 (cherry picked from commit 7041720ac208fa2f7f65e40d8133d4b291516847) Change-Id: I0cfab6aa588cd7e1a830d94f28ee17a8a6a68d98
2015-05-20Prefer /proc/<pid>/task/<pid>/maps over /proc/<pid>/maps on Linux.Jason Evans
This resolves #227. Bug: 21326736 (cherry picked from commit 8e33c21d2d03ee7f540e32c3d75b10c128eaea57) Change-Id: Ice5b86447aecbacff87f883d47d0df0adbba7f8c
2015-05-20Concise JEMALLOC_HAVE_ISSETUGID case in secure_getenv().Igor Podlesny
Bug: 21326736 (cherry picked from commit 95e88de0aab257020dfc33248b86331cbfac28b1) Change-Id: Iefa6d915943c5910618649b1ba45bc6ae8393d8b
2015-05-19Impose a minimum tcache count for small size classes.Jason Evans
Now that small allocation runs have fewer regions due to run metadata residing in chunk headers, an explicit minimum tcache count is needed to make sure that tcache adequately amortizes synchronization overhead.
2015-05-19Fix performance regression in arena_palloc().Jason Evans
Pass large allocation requests to arena_malloc() when possible. This regression was introduced by 155bfa7da18cab0d21d87aa2dce4554166836f5d (Normalize size classes.).
2015-05-19Fix nhbins calculation.Jason Evans
This regression was introduced by 155bfa7da18cab0d21d87aa2dce4554166836f5d (Normalize size classes.).
2015-05-15Avoid atomic operations for dependent rtree reads.Jason Evans
2015-05-06Implement cache index randomization for large allocations.Jason Evans
Extract szad size quantization into {extent,run}_quantize(), and . quantize szad run sizes to the union of valid small region run sizes and large run sizes. Refactor iteration in arena_run_first_fit() to use run_quantize{,_first,_next(), and add support for padded large runs. For large allocations that have no specified alignment constraints, compute a pseudo-random offset from the beginning of the first backing page that is a multiple of the cache line size. Under typical configurations with 4-KiB pages and 64-byte cache lines this results in a uniform distribution among 64 page boundary offsets. Add the --disable-cache-oblivious option, primarily intended for performance testing. This resolves #13.
2015-05-01Rename pprof to jeprof.Jason Evans
This rename avoids installation collisions with the upstream gperftools. Additionally, jemalloc's per thread heap profile functionality introduced an incompatible file format, so it's now worthwhile to clearly distinguish jemalloc's version of this script from the upstream version. This resolves #229.
2015-05-01Prefer /proc/<pid>/task/<pid>/maps over /proc/<pid>/maps on Linux.Jason Evans
This resolves #227.
2015-04-30Concise JEMALLOC_HAVE_ISSETUGID case in secure_getenv().Igor Podlesny
2015-04-22Temporary patch to avoid crashes.Christopher Ferris
Bug: 20296122 Change-Id: Ifabfa7a72f38b1781155624dc101464edb7a38be
generated by cgit v1.2.3 (git 2.25.1) at 2024-07-11 06:56:36 +0000