New AMD ERAPS Feature Yields Additional Performance Gains On Zen 5

At the beginning of November I wrote about AMD Linux engineers posting Linux patches enabling a new "ERAPS" feature for Zen 5. ERAPS wasn't talked about by AMD at the Zen 5 launches of the Ryzen 9000 / Ryzen AI 300 series or with the more recent EPYC 9005 "Turin" launch but when enabled, the Enhanced Return Address Prediction Security feature can help deliver some additional gains on new AMD Zen 5 systems by allowing some existing software security mitigations to be avoided. Here are some preliminary comparison benchmarks showing the benefit in affected workloads for using ERAPS on Linux with AMD Zen 5.

ERAPS is short for Enhanced Return Address Prediction Security. Besides not being talked about at the AMD Zen 5 launch events, it's also not currently covered by AMD's public official programming documentation but messages on the Linux kernel mailing list have indicated that ERAPS should soon be covered in their documentation. AMD ERAPS is for helping to recover some of the performance of security mitigations introduced following the Spectre class vulnerabilities over the past several years. ERAPS found with Zen 5 processors is a new defense for mitigating certain classes of speculative attacks such as Return Stack Buffer (RSB) poisoning attacks. In turn with ERAPS enabled some of the existing CPU security software mitigations can be relaxed, such as the explicit RET stuffing/filling on context switching.

The AMD ERAPS Linux patches explained:

"Remove explicit RET stuffing / filling on VMEXITs and context switches on AMD CPUs with the ERAPS feature (Zen5).

With the Enhanced Return Address Prediction Security feature, any hardware TLB flush results in flushing of the RSB (aka RAP in AMD spec). This guarantees an RSB flush across context switches. The feature also explicitly tags host and guest addresses - eliminating the need for explicit flushing of the RSB on VMEXIT.

The BTC_NO feature in AMD CPUs ensures RET predictions do not speculate from outside the RSB. Together, the BTC_NO and ERAPS features ensure no flushing or stuffing of the RSB is necessary anymore."

Since the original posting of the ERAPS Linux kernel patches earlier this month, the code continues to be discussed among upstream stakeholders. While the AMD ERAPS patches had appeared in tip/tip.git's x86/cpu branch ahead of the Linux 6.13 merge window, the patches were since removed from the "x86/cpu" branch. With the ERAPS patches having not returned to that branch or any other prominent branch and the Linux 6.13 merge window now underway, it doesn't appear that this AMD ERAPS support will be landing for Linux 6.13... Thus now looking at Linux 6.14 or later next year.

In any event I re-based the ERAPS v2 Linux patches atop v6.12 Git as of last week and proceeded to run some comparison benchmarks. I ran benchmarks of Linux 6.12 Git with all the default mitigations and then repeated the benchmarks on Linux 6.12 Git with the ERAPS v2 patches applied.

When on a patched Linux kernel with the ERAPS code applied and on a supported processor, as part of the "spectre_v2" mitigation reporting it will indicate "ERAPS hardware RSB flush" as active. While the patches have talked about ERAPS in the context of AMD EPYC Turin processors, I have verified that this ERAPS-patched kernel does work as well on Ryzen 9000 series desktop processors. So with more desktop-oriented benchmarks I'll be testing ERAPS this week to look for the performance impact on the desktop side -- today is focused on the server side with EPYC.

Using a Supermicro H13SSL-N server with EPYC 9655 processor I ran the benchmarks using Ubuntu 24.10 and then the mentioned Linux 6.12 with/without ERAPS patches. All other CPU security mitigations were at their defaults.