No announcement yet.

NVIDIA Improving Linux NUMA Distance Interface To Enhance Performance

  • Filter
  • Time
  • Show
Clear All
new posts

  • NVIDIA Improving Linux NUMA Distance Interface To Enhance Performance

    Phoronix: NVIDIA Improving Linux NUMA Distance Interface To Enhance Performance

    NVIDIA engineers have been working on NUMA distance metrics within the Linux kernel to replace the simple local/remote NUMA preference interface currently used by some drivers for NUMA-aware memory allocations. In their testing this improved NUMA distance handling is leading to "significant performance implications" for throughput and CPU utilization...

    Phoronix, Linux Hardware Reviews, Linux hardware benchmarks, Linux server benchmarks, Linux benchmarking, Desktop Linux, Linux performance, Open Source graphics, Linux How To, Ubuntu benchmarks, Ubuntu hardware, Phoronix Test Suite

  • #2
    Anyone able to decipher this? Please...


    • #3
      Originally posted by timofonic View Post
      Anyone able to decipher this? Please...
      Let's start with the patch comment quoted in the article:
      Implement and expose CPU spread API based on the scheduler's sched_numa_find_closest(). Use it in mlx5 and enic device drivers. This replaces the binary NUMA preference (local / remote) with an improved one that minds the actual distances, so that remote NUMAs with short distance are preferred over farther ones.

      This has significant performance implications when using NUMA-aware memory allocations, improving the throughput and CPU utilization.
      So, it sounds like the kernel's scheduler has a more nuanced view of NUMA, in its sched_numa_find_closest() function. They're enabling device drivers to use the same (or similar) underlying model, rather than a simplistic local vs. remote model.

      For machines with more than one level of hierarchy in their NUMA topology, this could be a win for locating resources closer to the device or perhaps the client userspace process than currently happens.

      Lastly, as they note it's from the Mellanox side of the house, whose products I expect are predominantly used on machines with multi-level NUMA.