Originally posted by david-nk
View Post
Announcement
Collapse
No announcement yet.
Jemalloc 5.3 Released With Many Speed & Space Optimizations
Collapse
X
-
Originally posted by C8292 View PostWhy isn't jemalloc default?
At minimum to become standard it would have to behave properly in that setting.
- Likes 2
Comment
-
Originally posted by guspitts View Post
As it stands, jemalloc does not always work properly when memory overcommit is disabled. That is often the case on cluster compute nodes. By experience, we had to disable jemalloc from computational biology software that would not run on a 1TB machine with jemalloc and no overcommit, whereas it would run with the standard glibc allocator and use very little memory. Not that jemalloc made the software use more memory, but it doesn't support no-overcommit well and would complain and crash. It is a known issue.
At minimum to become standard it would have to behave properly in that setting.
Comment
-
Originally posted by sinepgib View Post
Why do you disable overcommit? Do you handle OOM in a particular way?
There are downsides to no having overcommit. Technically, a process having more than 50% of memory could be denied a simple fork+exec. That does not seem happen in practice. Processes compiled with the address sanitizer (`-fsanitize=address`) do not work.
For us, not having overcommit made our compute nodes more stable. (I don't disable overcommit on my laptop on the other hand).
- Likes 3
Comment
-
Originally posted by bug77 View Post
That info needs more context to be relevant. It is possible for an allocator to reserve memory in advance when it detects frequent allocations. It will use more memory, but in doing so it may also dramatically improve throughput.
"More than good enough" is probably a reference to most apps that are not actually memory intensive.
i switched it jemalloc for it, and memory usage started staying under 3.5GB, as opposed to going well over 12GB.
- Likes 1
Comment
-
Originally posted by yoshi314 View Posti've had cases of apps leaking memory with glibc and not leaking it with jemalloc. the 'more than good enough' is seriously debatable (personally speaking).
or maybe it was not leaking but generally higher memory footprint. but the difference was truly dramatic.
There may be an inherent problem with glibc that you managed to encounter, but it's certainly not NOT "more than good enough" for a vast majority of cases. If that was true, we'd all be rebooting our servers (or at least, restarting all the userspace pieces) every week because of the memory leaks.
- Likes 2
Comment
-
Originally posted by bug77 View Post
That info needs more context to be relevant. It is possible for an allocator to reserve memory in advance when it detects frequent allocations. It will use more memory, but in doing so it may also dramatically improve throughput.
"More than good enough" is probably a reference to most apps that are not actually memory intensive.
Comment
-
Originally posted by bug77 View Post"More than good enough" is probably a reference to most apps that are not actually memory intensive.
With the per-thread malloc cache, the performance difference between jemalloc and the GLIBC allocator was almost eliminated.
Comment
-
Originally posted by sinepgib View Post
Why do you disable overcommit? Do you handle OOM in a particular way?Last edited by zboszor; 16 May 2022, 11:42 PM.
- Likes 1
Comment
Comment