Facebook Developing THP Shrinker To Avoid Linux Memory Waste

[phoronix]

Phoronix: Facebook Developing THP Shrinker To Avoid Linux Memory Waste

Meta/Facebook engineers have announced their work on THP Shrinker as a way for Linux's Transparent Hugepages (THP) to be more efficient and avoiding memory waste by removing under-utilized transparent hugepages...

https://www.phoronix.com/news/Linux-THP-Shrinker

[rabcor]

Oh that sounds pretty cool!

[milkylainen]

Freudian slip. I misread that as THP stinker...

[MorrisS.]

Which kernel will integrate this feature? What is the percentage of the improvement?

[cl333r]

Freudian slip. I misread that as THP stinker...

I almost joked about them putting the THP in cold water instead to get it to shrink.

[mangeek]

I'm a big fan of huge pages. I think we should probably be using them wherever possible except in the most restrictive environments. Internal kernel memory structures, the page cache, filesystem tuning for small file affinity within 1MB boundaries to optimize large-page block caching, swap... I'll bet there are a ton of efficiencies to be had when more and more of the overhead is handled in 1-2MB chunks instead of 4K at a time.

[JEBjames]

Michael

Typo/Grammar "Eventually they engineers are hoping that with the THP Shrinker, they" should be something else. Perhaps "Eventually their engineers are hoping that with the THP Shrinker they" would be a bit better.

[Mitch]

Excuse my ignorance, but am I right in guessing that this doesn't affect normal desktop usage such as Browsers, DE's, Videogames, Video + Photo Editing, engineering?

[CochainComplex]

tip for that pic

[mangeek]

Excuse my ignorance, but am I right in guessing that this doesn't affect normal desktop usage such as Browsers, DE's, Videogames, Video + Photo Editing, engineering?

THP in general does not, though Fedora|CentOS|RHEL have it enabled by default, and it did have a very rocky start and leave a lot of people with a bad impressions on the desktop side. Huge pages help make some workloads much more efficient, so an improvement to THP is good no matter what. If this change works well and mitigates some of the bad effects of huge pages (more memory consumption, allocation stalls), it'll probably open the door to enabling THP by default, including on desktops.