Red Hat Aiming To Address IBRS Mitigation Still Being Too Costly On Performance

Red Hat engineers are working to deal with Indirect Branch Restricted Speculation (IBRS) being too costly for mitigating Spectre V2 and Retbleed on older Intel Xeon Scalable processors. A new patch has been floated to disable IBRS when idle and is working out well at least for Red Hat Enterprise Linux 9 while isn't clear yet if it will be accepted into the upstream kernel.

The patch from Red Hat's Waiman Long explains the pains they are still dealing with around Intel IBRS for dealing with Spectre and Retbleed:

For Intel processors that need to turn on IBRS to protect against Spectre v2 and Retbleed, the IBRS bit in the SPEC_CTRL MSR affects the performance of the whole core even if only one thread is turning it on when running in the kernel. For user space heavy applications, the performance impact of occasionally turning IBRS on during syscalls shouldn't be significant. Unfortunately, that is not the case when the sibling thread is idling in the kernel. In that case, the performance impact can be significant.

When DPDK is running on an isolated CPU thread processing network packets in user space while its sibling thread is idle. The performance of the busy DPDK thread with IBRS on and off in the sibling idle thread are:

IBRS on IBRS off
------- --------
packets/second: 7.8M 10.4M
avg tsc cycles/packet: 282.26 209.86

This is a 25% performance degradation. The test system is a Intel Xeon 4114 CPU @ 2.20GHz.

This patch series turns off IBRS when in various idle mode to eliminate the performance impact of the idling thread on its busy sibling thread.

Ouch, a 25% hit on the Xeon Scalable Skylake for the open-source Data Plane Development Kit (DPDK).

There is this kernel thread where the patch to disable IBRS when idle is being floated. Prominent Intel Linux engineer Peter Zijlstra though has suggested another patch that isn't currently back-ported to RHEL9. Additionally, the possibility of going with Call Depth Stuff/Tracking instead of IBRS. In my tests the Call Depth Tracking found in mainline Linux 6.2+ is indeed offering to help recover some lost performance on Intel Skylake era CPUs that otherwise rely on IBRS. So we'll see where the upstream kernel activity goes or if Red Hat ends up just carrying this patch as part of their RHEL9 kernel for the time being until back-porting any new options. In any event this latest kernel mailing list thread continues to show the mitigation pains still being experienced by enterprise Linux users in mid-2023 on older platforms.