summaryrefslogtreecommitdiff
path: root/google/compression_utils.h
diff options
context:
space:
mode:
authorBenoît Lizé <lizeb@chromium.org>2019-02-14 21:33:28 +0000
committerCommit Bot <commit-bot@chromium.org>2019-02-14 21:33:28 +0000
commit7b94297364f4fa70e51086c59e971254cd2861e9 (patch)
treeb7ebace753553cd7810fee98dd438b5a1422958e /google/compression_utils.h
parent7fc69a35c16f98f8d2d9ab41f765077138858c50 (diff)
downloadzlib-7b94297364f4fa70e51086c59e971254cd2861e9.tar.gz
blink/bindings: Use PartitionAlloc for zlib's temporary data.
zlib uses malloc() to allocate temporary data. Unfortunately, on Android, jemalloc (malloc()'s underlying implementation) caches temporary data after free() in a thread-local cache. As renderers don't allocate from malloc() often in background threads, this data is not reclaimed. This increases memory usage by up to 256kiB per background compression thread, which is the cause for a sizable malloc() memory regression from foreground string compression (see linked bug). The deeper reason for the regression is more complex, see details in the linked bug, and https://docs.google.com/document/d/1aRIifaHF5l9relq9vHirTOWt2AZoAkDNqHiYZrWGE9A/edit?usp=sharing It involves some interaction between: - jemalloc implementation and memory accounting - Android's configuration of jemalloc - Chrome's allocation and threading patterns - zlib's allocation patterns Nevertheless, the regression is real, and to mitigate it, use PartitionAlloc to allocate zlib's temporary data. This is not necessary for decompression, as the allocation patterns are not the same, and the main thread doesn't have the same issues. As such, only do it for compression, to avoid a needlessly complex CL. On PartitionAlloc vs malloc(): The only path using partition alloc in this CL in from the renderer process, where it is already widely used. The issue is less likely to appear in the browser process as malloc() is more widely used there, though further investigation may reveal issues. Android vs everywhere: The issue is present at least with jemalloc(), that is Android. However thread caches and arena-allocators are used on other platforms as well, and as the allocation is not performance-sensitive, using PartitionAlloc everwhere makes the code simpler, and protects against potential issues (as we know that this specific issue does not arise with PartitionAlloc). Bug: 931553, 924164 Change-Id: I9e94e9cea5d51fac67b04fdb1681427e57bfbe1f Reviewed-on: https://chromium-review.googlesource.com/c/1472578 Commit-Queue: Benoit L <lizeb@chromium.org> Reviewed-by: Alexei Svitkine <asvitkine@chromium.org> Reviewed-by: Kentaro Hara <haraken@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#632372} Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src Cr-Mirrored-Commit: 5e7e1b1967f6d36206d0f18bcf25d99041bfdc37
Diffstat (limited to 'google/compression_utils.h')
-rw-r--r--google/compression_utils.h6
1 files changed, 5 insertions, 1 deletions
diff --git a/google/compression_utils.h b/google/compression_utils.h
index e84bfc4..fe741d0 100644
--- a/google/compression_utils.h
+++ b/google/compression_utils.h
@@ -15,11 +15,15 @@ namespace compression {
// |output_buffer|, of size |output_buffer_size|. If the buffer is large enough
// and compression succeeds, |compressed_size| points to the compressed data
// size after the call.
+// |malloc_fn| and |free_fn| are pointers to malloc() and free()-like functions,
+// or nullptr to use the standard ones.
// Returns true for success.
bool GzipCompress(base::StringPiece input,
char* output_buffer,
size_t output_buffer_size,
- size_t* compressed_size);
+ size_t* compressed_size,
+ void* (*malloc_fn)(size_t),
+ void (*free_fn)(void*));
// Compresses the data in |input| using gzip, storing the result in |output|.
// |input| and |output| are allowed to point to the same string (in-place