Linux Adding New Control Since Its Splitlock Detector Is Wrecking Some Steam Play Games

Back in 2020 the Linux kernel added a split-lock detector since when they occur an atomic instruction spanning multiple cache lines and requiring a global bus lock is needed. This has an unfortunate heavy impact on the system and thus the detector was added to report it to the kernel log when a split-lock occurs. But earlier this year starting with Linux 5.19, kernel developers decided to "make life miserable" and intentionally slow down bad behaving apps that abuse split-locks. That in turn has caused problems for some games -- so far select Windows games running under Steam Play -- and thus a new kernel knob is being added to more easily adjust the behavior.

Split locks can take at least one thousand more cycles than an atomic operation within a single cache line. Splitlocks can hurt overall system performance and thus with the Linux 5.19 kernel they wanted to "make life miserable for split lockers" by forcing sequential access for user-space split locks. This hurts the performance of software leading to the split locks but can help maintain overall system performance.

"Make life miserable for apps using split locks by slowing them down considerably while the rest of the system remains responsive. The hope is it will hurt more and people will really fix their misaligned locks apps."

While it's nice in theory and can incentivize open-source developers to improve their code after users notice the slowdown and messages in the kernel log, it's less easy to improve the situation when it comes to proprietary software... Especially commercial Windows games running on Linux by way of Steam Play. Or legacy closed-source software that is no longer maintained.

GitHub user "pibberflibbits" initially reported a DXVK bug over very low performance with the game God of War, but the awful performance turned out to be due to an intentional kernel change around its split-lock detector and this proprietary Windows game abusing split-locks. With Linux 6.2+ will at least be a means of more easily changing the kernel behavior.

One title that was found to be abusing splitlocks is the game God of War. It turns out when running on Linux 5.19+ there is "insanely low performance" and it turned out to be due to the split-lock change. If booting the kernel with "split_lock_detect=off" the God of War performance on Linux improves greatly.

It's likely there are some other games out there too with the splitlock issue, but the notable God of War game is what led to kernel developers now refining their split-lock detection behavior.

A new sysctl tunable of "split_lock_mitigate" is now added for Linux 6.2 to easily disable the "misery mode" behavior. When setting split_lock_mitigate to 0, there will just be the warning in the kernel lock but not the sequential access penalty. By default that intentional performance degradation mode is active still, but presumably we'll see SteamOS and the like set the split_lock_mitigate tunable to off automatically. It's also possible we'll see this automatically adjusted with the likes of Feral's GameMode. With it being a sysctl option it's easier to now manage at run-time rather than having to use the split_lock_detect option for the kernel at boot time, but the kernel developers so far want to keep the default "miserable performance" mode to motivate user-space software improvements around reducing split-locks.

This includes a single change which enables users to reduce the penalty inflicted on split lock users. There are some proprietary, binary-only games which because entirely unplayable with the old penalty.

The change was sent in as the x86/splitlock material for Linux 6.2.