Speculative NUMA Fault Support Proposed For Improving Tiered Memory Linux Performance

This year there has been a lot of Linux kernel work around improving the handling of tiered memory servers, namely those with traditional system RAM augmented by Intel Optane DC persistent memory. There has been work to demote pages during reclamation to the slower persistent memory, improving NUMA balancing around such systems to optimize memory placement, transparent page placement and related work around tiered memory Linux servers.

The latest work comes by way of Alibaba and they are proposing speculative NUMA fault support for tiered memory systems. The proposed patches for speculative NUMA fault support will handle promotion of slow/hot memory to fast memory for enhancing system performance.

This RFC patch set adds speculative numa fault support for some scenarios, like tiered memory system. On tiered memory system, it will rely on the numa balancing to promote slow and hot memory to fast memory to improve performance. So we can promote several sequential pages on slow memory in advance according to the data locality for some workloads to improve the performance.

So now how much pages need to be promoted to fast memory is the best? Now this RFC patch set only implements a basic and simple mechanism to speculate the numa fault window for each VMA. It will introduce a new atomic member for each VMA to record the numa fault window information, which is used to determine if it is a sequential stream to expand or reduance the numa fault window.

The patch series is under a "request for comments" flag and amounts to dealing with just over 200 lines of memory management code at this point. The results are promising with this basic/simple mechanism already showing around a 6% improvement to performance for MySQL on a tiered memory server.

It's great seeing these tiered memory Linux performance optimizations continuing ahead of next-generation servers where we are expecting to see greater use of persistent memory.