RDMA Changes For Linux 5.12 Add DMA-BUF Support For Peer-To-Peer Transfers With GPUs

Written by Michael Larabel in Hardware on 22 February 2021 at 12:09 PM EST. 11 Comments
The changes within the remote direct memory access (RDMA) subsystem for Linux 5.12 are deemed "quite small" but there is one interesting addition courtesy of Intel.

Covered last year on Phoronix was the Intel-led work on DMA-BUF support for RDMA for supporting peer-to-peer transactions over PCI Express between RDMA-enabled NICs and other PCIe devices, such as graphics cards. That peer-to-peer support via DMA-BUF with a focus on GPU support is what's coming in Linux 5.12.

The RDMA pull for Linux 5.12 includes the support for creating an RDMA memory region from a DMA-BUF ID to allow for PCIe peer-to-peer transfers with GPU video memory. This work will presumably play a role with Intel's forthcoming Xe HP/HPC offerings in more efficient transfers to/from RDMA devices.

The RDMA peer-to-peer support should yield better performance than going through system memory and avoiding excess copies. There is already P2PDMA and Heterogeneous Memory Management (HMM) within the kernel for dealing with similar use-cases while this new addition to the RDMA code for DMA-BUF importing using device memory should be a simpler alternative, doesn't require page structure, and more easily adapt to existing GPU drivers. The usage for now may be a bit limited though until the GPU drivers allow pinning device memory via the DMA-BUF interface, as Intel engineers noted in the original patches.

The RDMA pull for Linux 5.12 also includes the usual assortment of driver updates, code clean-up, support for more than 255 ports on an RDMA device, and other changes.
Related News
About The Author
Michael Larabel

Michael Larabel is the principal author of Phoronix.com and founded the site in 2004 with a focus on enriching the Linux hardware experience. Michael has written more than 20,000 articles covering the state of Linux hardware support, Linux performance, graphics drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org automated benchmarking software. He can be followed via Twitter, LinkedIn, or contacted via MichaelLarabel.com.

Popular News This Week