NVIDIA Improving Linux NUMA Distance Interface To Enhance Performance

NVIDIA engineers have been working on NUMA distance metrics within the Linux kernel to replace the simple local/remote NUMA preference interface currently used by some drivers for NUMA-aware memory allocations. In their testing this improved NUMA distance handling is leading to "significant performance implications" for throughput and CPU utilization.

This NVIDIA work isn't part of their graphics driver effort but rather on the Mellanox networking side of the house. Tariq Toukan summed it up in the latest spin of these kernel patches:

Implement and expose CPU spread API based on the scheduler's sched_numa_find_closest(). Use it in mlx5 and enic device drivers. This replaces the binary NUMA preference (local / remote) with an improved one that minds the actual distances, so that remote NUMAs with short distance are preferred over farther ones.

This has significant performance implications when using NUMA-aware memory allocations, improving the throughput and CPU utilization.

So far the results with the Mellanox and ENIC network drivers are looking very good from NVIDIA's testing on AMD EPYC 7763 servers:

See this patch series for more details.