NVIDIA Engineer Shows Off New Linux Patches For Proactive Memory Compaction

Veteran Linux engineer Nitin Gupta of NVIDIA has unveiled his latest patches on the work he got started on last year: proactive memory compaction for Linux motivated by the latency issues brought on by he current on-demand compaction when an application requests a lot of hugepages.

Gupta began publishing the proactive compaction work for the Linux kernel towards the end of 2019. The work is particularly aimed at applications requesting lots of memory as hugepages but has other benefits as well for systems with fragmented RAM.

Today he published the v2 proactive compaction patches. With this new version there are new tunables for per-node and per-zone compaction scores. Other code improvements were also made. The proactiveness score for this code still remains controlled via sysfs so depending upon system preferences, the effectiveness of this code can be controlled.

In one of his tests when running Java with transparent huge-pages enabled, to allocate 700G of Java heaps using hugepages the time dropped from 27 minutes to just over three minutes, among other promising numbers.

Hopefully this Linux memory management improvement will go through its final rounds of review punctually and ideally see the mainline tree in the near future.