XFS Improvement For Linux 5.20 Enhances Scalability For Large Core Count Systems

One of several improvements being prepared for the XFS file-system with the upcoming Linux 5.20 cycle is focused on improving the CIL scalability for systems with many CPU cores.

XFS' Darrick Wong has been working to improve the CIL scalability and has merged this work into xfs-5.20-merge ahead of the Linux 5.20 merge window opening up at the end of the month.

Here's the summary of this particular scalability improvement:

This series aims to improve the scalability of XFS transaction commits on large CPU count machines. My 32p machine hits contention limits in xlog_cil_commit() at about 700,000 transaction commits a section. It hits this at 16 thread workloads, and 32 thread workloads go no faster and just burn CPU on the CIL spinlocks.

This patchset gets rid of spinlocks and global serialisation points in the xlog_cil_commit() path. It does this by moving to a combination of per-cpu counters, unordered per-cpu lists and post-ordered per-cpu lists.

This results in transaction commit rates exceeding 1.4 million commits/s under unlink certain workloads, and while the log lock contention is largely gone there is still significant lock contention in the VFS (dentry cache, inode cache and security layers) at >600,000 transactions/s that still limit scalability.
...
Overall, this pushes the transaction commit bottleneck out to the lockless reservation grant head updates. These atomic updates don't start to be a limiting fact until > 1.5 million transactions/s are being run, at which point the accounting functions start to show up in profiles as the highest CPU users. Still, this series doubles transaction throughput without increasing CPU usage before we get to that cacheline contention breakdown point...

This scalability improvement is good news for those on large Threadripper / Core i9 / Xeon / EPYC platforms running the XFS file-system. The xfs-5.20-merge is also collecting a number of other feature additions and improvements for introduction in the upcoming merge window.