Linux 6.0 Merges The AMD Performance Fix For The Old "Dummy Wait" Workaround

This morning I called attention to some pending work around a 20 year old chipset workaround in the Linux kernel had been hurting modern AMD systems by erroneously still applying the change to modern hardware. Fortunately, that patch has now been picked up by Linus Torvalds in time for the Linux 6.0 kernel expected for its stable debut next weekend.

As outlined in that earlier article, since 2002 when ACPI support was added to the kernel there was a "dummy wait" operation added due to some chipsets at the time where STPCLK# wasn't asserted in time along the idle path in the kernel. The dummy I/O read delays further instruction processing until the CPU is fully stopped. But an AMD engineer recently noticed this behavior being applied on modern AMD Zen 3 hardware and finding that it could lead to performance issues for workloads rapidly switching between busy and idle phases and especially for larger core count systems like Ryzen Threadripper and EPYC platforms.

Sampling certain workloads with IBS on AMD Zen3 system shows that a significant amount of time is spent in the dummy op, which incorrectly gets accounted as C-State residency. A large C-State residency value can prime the cpuidle governor to recommend a deeper C-State during the subsequent idle instances, starting a vicious cycle, leading to performance degradation on workloads that rapidly switch between busy and idle phases.

One such workload is tbench where a massive performance degradation can be observed during certain runs.

AMD engineer K Prateek Nayak showed the significant performance impact that this erroneous workaround for modern hardware can have on AMD systems. Intel systems meanwhile don't use this code path for modern hardware and thus unaffected.

An AMD patch was originally suggested but then cleaned-up/simplified by Intel engineer Dave Hansen. That patch simply doesn't apply this "dummy wait" workaround except for older (pre-Nehalem) Intel systems and thus AMD systems will now forego this operation that can degrade performance on modern systems. With it mostly impacting workloads switching often between busy and idle states plus more noticeable for larger core count systems, AMD EPYC server performance with this patch should be quite interesting especially for web server / database workloads and other type rapid tests. I'll be firing up a complete set of wide-ranging benchmarks evaluating this patch tomorrow.

The patch was mainlined this evening as part of the "x86/urgent" fixes sent in as part of this pull ahead of Linux 6.0's expected stable release on 2 October. Great to see it land quickly and stay tuned for some benchmarks.