Device Memory TCP Nears The Finish Line For More Efficient Networked Accelerators

Written by Michael Larabel in Linux Networking on 8 July 2024 at 08:13 AM EDT. 3 Comments
A year ago Google engineers posted experimental Linux code for Device Memory TCP for more efficient transferring of data from GPUs/accelerators to network devices without having to go through a host CPU memory buffer. After going through many rounds of review, Device Memory TCP appears to be nearing the finish line.

Device Memory TCP "Devmem TCP" is a Linux kernel feature being baked to allow transferring data to and/or from device memory efficiently without having to bounce the data through a host memory buffer. Due to high memory/network bandwidth requirements particularly for AI training with many interconnected systems and relying on TPUs / GPUs / NPUs / other accelerator device types in general, the goal has been to avoid memory copies through the host system memory when sending or receiving data from those discrete devices across the network.

Device Memory TCP allows for more efficient network communication

Device Memory TCP brings socket APIs to allow sending device memory across the network directly and to receive incoming network packets directly into device memory. This both helps with avoiding host memory bandwidth pressure as well as reducing PCI Express bandwidth pressure in not having to go through the PCIe root complex.

It appears Device Memory TCP is wrapping up with the prep patches having been queued last week into the networking subsystem's "net-next.git" tree. Thus the prep patches for Device Memory TCP at least will land for Linux 6.11. There's still a week or two to see if the Device Memory TCP work itself will be queued up too into net-next ahead of the v6.11 merge window, otherwise it's looking like it will arrive for v6.12 which should be exciting given that it will likely be the 2024 LTS kernel version.
Related News
About The Author
Michael Larabel

Michael Larabel is the principal author of and founded the site in 2004 with a focus on enriching the Linux hardware experience. Michael has written more than 20,000 articles covering the state of Linux hardware support, Linux performance, graphics drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and automated benchmarking software. He can be followed via Twitter, LinkedIn, or contacted via

Popular News This Week