Heterogeneous Memory System (HMS) Prototype Published For The Linux Kernel

The past several years Red Hat developer Jerome Glisse has been working on Heterogeneous Memory Management (HMM) for the Linux kernel to handle the mirroring of process address spaces, system memory that can be transparently used by any device process, and similar functionality around today's GPU computing needs and other devices. Jerome today published the next step as part of his low-level memory device management work and that is the Heterogeneous Memory System for exposing complex memory topologies of today's systems.

Under a "request for comments" flag, Jerome Glisse today published his work at Red Hat on the Heterogeneous Memory System (HMS) and a new hbind() system call. HMS seeks to better expose complex system topology configurations and different kinds of device memory to user-space. This work also provides a new user-space API to bind/hint to the kernel about what memory to use for a range of virtual address space, and the kernel-side changes for the virtual memory policy to deal with HMS.

Among the information exposed by HMS in a standardized manner would be the memory target (RAM, HBM, etc), the associated device or CPU, the type of link between the memory and the device, any bridge(s) between links, and properties like the memory bandwidth/latency and other information relevant for programs in determining their desired memory placement for a given workload.

Besides plumbing in the proposed HMS infrastructure to the Linux kernel, Jerome also prototyped a deployment for the Nouveau DRM (open-source NVIDIA) graphics driver for exposing the GPU memory.

For much more information on the Heterogeneous Memory System proposal, see the patch letter sent out this evening on the kernel mailing list. The documentation patch also has additional technical detail. It will likely be a while before HMS is settled upon and upstream, but should be useful for future GPGPU programming and other use-cases.