Intel Sends Out Latest AMX Support For The Linux Kernel

Of all the new Linux 5.11 features and all the enablement work Intel has already completed for Xeon "Sapphire Rapids" hardware, one big feature not yet mainlined is the Advanced Matrix Extensions (AMX) support.

Since this past summer Intel has been posting open-source patches around AMX from the compiler toolchains to the kernel support. Ahead of EOY, Intel's engineers have sent out the latest AMX kernel patches for Linux.

AMX is Intel's new programming paradigm with a focus on better AI performance both for training and inference. AMX is built around the concept of "tiles" as a set of two-dimensional registers for representing a larger memory image and accelerators that can operate on said tiles. Initial AMX features are for BFloat16, TILE, and INT8 while the design is extensible for new accelerators to be added later.

Yes, like AVX, making use of AMX can impact the performance of any other workloads on a given core / sibling thread. "Further, both technologies run faster when they are not simultaneously running on SMT siblings, and both technologies use of power and bandwidth impact the power and performance available to neighboring cores. (This impact has measurably improved in recent hardware.)"

With the third version of the Intel AMX patches sent out last week, there is improved test code, simplified command line handling of AMX options, and a number of low-level code changes.

Those curious about the low-level workings of AMX from the kernel side can see the v3 patches. With the Linux 5.11 merge window now over, this work obviously won't come this cycle but not until at least Linux 5.12 in the spring. But given that Sapphire Rapids hardware won't be out until the very end of 2021 or slip into 2022, there still is sufficient time for this big feature to get ironed out in the Linux/open-source landscape ahead of launch.