AMD Ryzen Threadripper 3970X / 3960X Linux Benchmarks

Considering the trouble-free experience of the AMD EPYC Rome processors and prior to that the Ryzen 3000 series working well on Linux once the systemd+RdRand issue was resolved, I wasn't expecting any Linux problems to come up at all with the TRX40 motherboards and Threadripper 3960X/3970X. But to much dismay there was one show-stopping problem.

With either the ASUS or MSI motherboards and with both the 3960X and 3970X, Linux distributions like Ubuntu 19.10 weren't booting! At first I thought perhaps the systemd+RdRand problem came up again, but nope, that was not the case. After trying some other combinations and looking at the boot text, there were signs pointing to an MCE (Machine Check Exception) problem.

Thankfully there is the "mce=off" kernel command line parameter to disable the MCE kernel code. So I tried that and sure enough the systems now booted without any problems!

When reaching out to AMD's Linux engineers about it, on 21 November they did release an AMD x86 MCE patch for the Linux kernel. Unfortunately as of writing that patch has yet to be picked up by the mainline Linux kernel or found by any distribution kernels for that matter. So, sadly, on launch-day if trying to easily boot up Linux distributions on the hardware you are likely to see a hang or MCE errors. But at least there is the "mce=off" parameter to workaround this issue until the MCE fix has landed.

Sadly this isn't the first Ryzen launch with Linux headaches at launch. For those that recall the original Ryzen launch there was the "performance marginality problem" affecting some Linux users, having working AMD thermal driver support on Linux was much belated originally and then also late for Zen+, and with the Zen2 launch in July was the big systemd+RdRand woes.

With AMD continuing to gain marketshare on both client and servers, hopefully in 2020 we'll see them investing in more Linux/open-source initiatives. The AMD EPYC Linux support has always been strong at launch unlike the Ryzen launches, but they are still stretched thin when it comes to Linux engineering resources on the CPU side with often seeing patches come via their partners at SUSE, Code Sourcery, etc. A decade ago back when there was the AMD Operating System Research Center they were more engaged with upstream Linux and hopefully with the continued Ryzen/EPYC growth we'll be able to see them allocate said resources to ensure solid Linux support at launch moving forward. Besides the launch headaches, there is still more that could be done to Znver2 tuning in the LLVM Clang and GCC compilers, there still isn't mainline support for being able to read the CPU power consumption on Zen CPUs (but there is the out-of-tree Zenpower driver), the BSD support at times is a bit iffy, some CPU features like SEV took a while past hardware availability to get mainlined (or another random example, even RDPRU reporting to /proc/cpuinfo is only coming now with the Linux 5.5 cycle), and other areas they could further polish their Linux/open-source software support.

But at least with that MCE issue out of the way, in my testing over the past two weeks, the Linux experience on the likes of Ubuntu 19.10 with Linux 5.3~5.4 kernels has been smooth. I'll have cross-distribution benchmarks and other metrics in the days ahead.