Linus Torvalds Just Made A Big Optimization To Help Code Compilation Times On Big CPUs

For those using GNU Make in particular as their build system, the parallel build times are about to be a lot faster beginning with Linux 5.6 for large core count systems. This landing just after the AMD Threadripper 3990X 64-core / 128-thread CPU launch is one example of systems to benefit from this kernel change when compiling a lot of code and making use of many GNU Make jobs.

Linus Torvalds himself changed around the kernel's pipe code to use exclusive waits when reading or writing. While this doesn't mean much for traditional/common piping of data, the GNU Make job-server is a big beneficiary as it relies upon a pipe for limiting the parallelism. This technique though employed by the GNU Make job server is inefficient with today's high core count CPUs as all of the spawned processes are woken up rather than a single reader to be woken upon a writer's release.

In a simplified test case written by Linus Torvalds, this patch caused the number of context switches on the test program to drop from 11 million to just 1.2 million. The consumed system time was also a tiny fraction of the original time. Reducing the number of context switches is also welcome simply for the fact of how much slower context switching performance has become on the Intel side as a result of the countless security mitigations.

Besides this patch landing now for the Linux 5.6 kernel, the patch was partially delayed as this improvement could actually trigger a race condition with the GNU Make job server, which ended up being resolved back in 2017 but only saw a new release with GNU Make 4.3 and then back-ported on some distributions.

Intel's Josh Triplett tested the patch with Linus and confirmed, "I've been hammering on your pipe fix patch (switching to exclusive wait queues) for a month or so, on several different systems, and I've run into no issues with it. The patch *substantially* improves parallel build times on large (~100 CPU) systems, both with parallel make and with other things that use make's pipe-based jobserver."

The patch is here for those interested from today's mainline Git state.

I'll certainly be testing this impact too on Make-based compilation performance as part of my Linux 5.6 benchmarking. It will be interesting to see the impact on GNU Make based projects, especially with many projects in recent years switching away from the GNU Build System over complaints of slow performance (as well as other factors) in relation to the likes of Meson+Ninja, but at least a lot of code out there is still dependent upon Make for this Linux 5.6+ change to benefit.