Facebook Developing THP Shrinker To Avoid Linux Memory Waste

[Alex/AT]

As THP literally helps only a few workloads and can heavily hurt most other, this is one other thing that should be disabled by default.

[halo9en]

Arch also has THP enabled by default. However, unless you're running a server...

[Alex/AT]

Arch also has THP enabled by default. However, unless you're running a server...

Not 'unless you're running a server', but unless you 'benchmark it versus idiotic^W synthetic condition of static 1GB (!!!) page size in 2-VM environment'.

[yump]

THP in general does not, though Fedora|CentOS|RHEL have it enabled by default, and it did have a very rocky start and leave a lot of people with a bad impressions on the desktop side. Huge pages help make some workloads much more efficient, so an improvement to THP is good no matter what. If this change works well and mitigates some of the bad effects of huge pages (more memory consumption, allocation stalls), it'll probably open the door to enabling THP by default, including on desktops.

I'm on Fedora and /sys/kernel/mm/transparent_hugepage/enabled says "madvise". I'm pretty sure that means programs don't get huge pages unless they explicitly request them, so not exactly "transparent". More transparent than statically assigned hugetlbfs, I suppose.

Not 'unless you're running a server', but unless you 'benchmark it versus idiotic^W synthetic condition of static 1GB (!!!) page size in 2-VM environment'.

Huge pages are great for anything with a big working set and substantially random access pattern. The game Factorio, in particular, absolutely loves huge pages. Apparently the best way is to LD_PRELOAD mimalloc in place of glibc malloc and tell it to use huge pages with an environment variable.

[Alex/AT]

Huge pages are great for anything with a big working set and substantially random access pattern. The game Factorio, in particular, absolutely loves huge pages. Apparently the best way is to LD_PRELOAD mimalloc in place of glibc malloc and tell it to use huge pages with an environment variable.

This completely fits into 'literally helps only a few workloads'.
And yes, madvise is indeed an ok way to use these. It's nice distros are starting to adopt exactly that way of doing things for problematic stuff.