Linux 5.19 To "Make Life Miserable" In Slowing Down Bad Behaving Split-Lock Apps
Back in 2020 Intel engineers working on the Linux kernel added split lock detection to provide a warning when an atomic operation spans multiple cache lines and requires a global bus lock for atomicity. A warning is now deemed not useful enough so instead the intent moving forward is to "make life miserable" for such misbehaving user-space applications by slowing down the performance with hopes of the app developers better handling their code.
While the default behavior is to provide a warning for user-space applications abusing split locks, the prior patches do allow killing the apps. Split locks can take at least 1,000 more cycles than an atomic operation within a single cache line. Split locks result in other CPUs being blocked and other bus agents for control are blocked as well -- affecting overall system performance.
This Linux split lock detection has relied on modern x86_64 CPU features to be able to generate an alignment check exception on hitting a split lock. The existing kernel behavior can be tuned using the "split_lock_detect=" kernel parameter.
Intel engineer Tony Luck proposed making things more aggressive and "make life miserable for split lockers" with patches posted earlier this year. The patches now queued for Linux 5.19 are trying to slow down the offending application so much so that the developer(s) will hopefully address the problem. The new "sequential" mode is forcing sequential access for user-space split locks. In turn this mode also ensures the overall system performance is better in these conditions.
That "make life miserable" change was submitted as the x86 split-lock updates for Linux 5.19.
While the default behavior is to provide a warning for user-space applications abusing split locks, the prior patches do allow killing the apps. Split locks can take at least 1,000 more cycles than an atomic operation within a single cache line. Split locks result in other CPUs being blocked and other bus agents for control are blocked as well -- affecting overall system performance.
This Linux split lock detection has relied on modern x86_64 CPU features to be able to generate an alignment check exception on hitting a split lock. The existing kernel behavior can be tuned using the "split_lock_detect=" kernel parameter.
Intel engineer Tony Luck proposed making things more aggressive and "make life miserable for split lockers" with patches posted earlier this year. The patches now queued for Linux 5.19 are trying to slow down the offending application so much so that the developer(s) will hopefully address the problem. The new "sequential" mode is forcing sequential access for user-space split locks. In turn this mode also ensures the overall system performance is better in these conditions.
Hence a new option to "slow it down in a way which makes it annoying".
Primary reason for this change is to provide better quality of service to the rest of the applications running on the system. Internal testing shows that even with many processes splitting locks, performance for the rest of the system is much more responsive.
Add a new choice to the existing "split_lock_detect" boot parameter "sequential". In this mode split lock detection is enabled. When an application tries to execute a bus lock the #AC handler.
1) Blocks (interruptibly) until it can get the semaphore
If interrupted, just return. Assume the signal will either kill the task, or direct execution away from the instruction that is trying to get the bus lock.
2) Disables split lock detection for the current core
3) Schedules a work queue to re-enable split lock detect in 2 jiffies
4) Returns
That "make life miserable" change was submitted as the x86 split-lock updates for Linux 5.19.
29 Comments