Jemalloc 5.3 Released With Many Speed & Space Optimizations

[phoronix]

Phoronix: Jemalloc 5.3 Released With Many Speed & Space Optimizations

Last week marked the first update to the jemalloc memory allocation library since August of 2019. This malloc() implementation focused concurrency and memory fragmentation avoidance has seen more speed optimizations and other improvements in this new jemalloc 5.3 release...

https://www.phoronix.com/scan.php?page=news_item&px=Jemalloc-5.3-Released

[C8292]

Why isn't jemalloc default?

[zboszor]

Why isn't jemalloc default?

Because the jemalloc developers chose to keep their code separate with using LD_PRELOAD instead of merging it into GLIBC. Also, the GLIBC allocator is more than good enough for many use cases.

[NobodyXu]

Why isn't jemalloc default?

Jemalloc uses a different LICENSE from glibc https://github.com/jemalloc/jemalloc/blob/dev/COPYING

[bug77]

Why isn't jemalloc default?

I have a feeling that's not an issue in this case. I would hope anyone that can understand what an allocator does and why they'd need one or the other, will be savvy enough to know how to replace them.
Also, if I'm not mistaken, jemalloc was the default allocator in Rust. But it was made optional at some point when it became apparent having a default allocator was too much overhead for some scenarios. Yup, there it is: https://internals.rust-lang.org/t/je...d-library/8759

[ssokolow]

Why isn't jemalloc default?

It is the default... on FreeBSD and NetBSD.

That said, to answer the intent of your question, allocators always make trade-offs and different OS devs weight different concerns differently.

[V1tol]

Redis on Linux uses jemalloc by default. At least package in my distro has it as required dependency.

[yoshi314]

Because the jemalloc developers chose to keep their code separate with using LD_PRELOAD instead of merging it into GLIBC. Also, the GLIBC allocator is more than good enough for many use cases.

i've had cases of apps leaking memory with glibc and not leaking it with jemalloc. the 'more than good enough' is seriously debatable (personally speaking).

or maybe it was not leaking but generally higher memory footprint. but the difference was truly dramatic.

[bug77]

i've had cases of apps leaking memory with glibc and not leaking it with jemalloc. the 'more than good enough' is seriously debatable (personally speaking).

or maybe it was not leaking but generally higher memory footprint. but the difference was truly dramatic.

That info needs more context to be relevant. It is possible for an allocator to reserve memory in advance when it detects frequent allocations. It will use more memory, but in doing so it may also dramatically improve throughput.

"More than good enough" is probably a reference to most apps that are not actually memory intensive.

[david-nk]

I used jemalloc 3 a lot and managed to achieve huge memory savings for some applications. However, jemalloc 5 doesn't seem that great anymore. It seems to use way more memory for minuscule performance gains.

Also, it has some pretty bad behaviors that cause it to retain memory far longer than it should.
For example, if you have 8 worker threads processing images and each of them, at some point, loads an image that uses up 10 GB of memory, that means the application will keep 80 GB allocated forever. So jemalloc is no longer a good default option like it might have been a few years ago.