Facebook Developing THP Shrinker To Avoid Linux Memory Waste
THPs are great for efficiency with fewer TLB cache misses for some workloads, but the 2MB vs. 4kb page size can lead to a lot of wasted memory if not being effectively utilized. Facebook engineers found about 2.7GB of memory waste on one of their platforms from under-utilized THPs.
With the THP Shrinker they are working on for the Linux kernel, they aim to address that waste while still enjoying THPs for greater CPU efficiency. The THP Shrinker will split the least utilized hugepages.
Facebook engineers are working to add additional tuning around this THP Shrinker and potentially tied into CPU/IO/Memory pressure events. Eventually the engineers are hoping that with the THP Shrinker, they could just deprecate Linux's madvise THP mode and switch to THP being enabled for all applications.
More details on this interesting kernel development via lore.kernel.org. Right now this THP Shrinker is just a little more than 500 lines of new kernel code.