Fast Kernel Headers v2 Posted - Speeds Up Clang-Built Linux Kernel Build By ~88%

[skeevy420]

*cough* ..as a German I'm obliged to point out that there a plenty of high quality options available. Just to name the obvious ones: Porsche, Mercedes, BMW, VW ....and then there are the acquired brands running on german engines: Lamborghini, Bugatti...but with the later ones you have issues with the shipmenttime aswell.

Ford "invented" the assembly line with high speed, mass produced vehicles and, because of that, was able to ship out vehicles 88% faster.

[kreijack]

See also: https://en.cppreference.com/w/cpp/language/modules

Apart the fact that the modules are for the C++ language, in the first patches set Ingo Molnar reported:

[...]
As to other approaches attempted: I also tried enabling the kernel to use pre-compiled headers (with GCC), which didn't bring a performance improvement beyond a 1-2% for repeat builds. Plus during development most of the precompiled headers are cache-cold in any case and need to be generated by the compiler again and again.
[...]

https://lwn.net/ml/linux-kernel/[email protected]/

So it seems that the increase of speed is due mostly to the header re-organization.

[sinepgib]

Why would it change runtime speed? This is about changing the include hierarchy. There is zero chance it will have a runtime impact.

Because it also changed some functions to not be suggested for inlining. Depending on whether the compiler decided to follow the hint and whether that hint was clever or not, that could make runtime performance slightly better or slightly worse. I doubt the change would be huge tho.

[oleid]

Are you planning already some benchmark comparison between the original kernel and the fast header version ? Would be nice to confirm it doesn't generaWhy would runtime speed change? This is Scott shifting code in headsets.
te any slowdown a runtime.

Why would it change runtime speed? This is about changing the include hierarchy. There is zero chance it will have a runtime impact.

edit: we did something similar a few years ago, albeit with a c++ codebase: Find out what headers get included all the time and slim them down (by including less and replacing actual class definitions with forward declarations; the definitions are most of the time only needed in the cpp file).

[perpetually high]

Just gave the patchset a whirl. Freakin incredible.

Cut my kernel build time on the 5800X from roughly 7min 10s seconds to 4min 40s

Ingo, you brilliant son of a gun! That's seriously an incredible effort that is going to be pay huge dividends over time.

Here's the git diff I used to make the patch if anyone needs:

$ git diff 391ce485ced0e47bf8d2ce8bc32bb87887e16656 c9e6606c7fe92b50a02ce51dda82586ebdf99b48 > fast_kernel_headers-v2.patch
$ patch -p1 < ./fast_kernel_headers-v2.patch

Should see CONFIG_FAST_HEADERS already enabled in the config. It did fail on "nfsd" which Is NFS server support, so I disabled that from module to disabled in the config since I'm not using it. I'm also getting a failure on linux/device_api_lock.h not being found when compiling the linux-tools package, usbip in particular. Not sure what's going on there, but this is exciting stuff.

[coder]

can we get 1 bargraph ?

Exactly.

I just click with the hope that the article would actually have at least some build benchmarks, if not also a few runtime tests.

[coder]

Off topic, but I'm constantly amazed to keep even experienced developers that fail to grasp two simple things (imho):
1. the more featured a language is, the more it takes to compile; you want generics, type inference, even inheritance, named parameters, be prepared to pay a price
2. longer compile times are always justified when they result in faster production code (which is almost all the time); languages like Java or C# may appear fast to compile, but they do not link anything - you'll pay a hefty runtime penalty for that

More on topic, the biggest expectation I have from these patches, whenever they manage to land, is that shorter build times will make fuzzing more feasible, so a lot more bugs will be exposed and patched.

That's a bit simplistic. I'm not sure there aren't a few things in C that needlessly make work harder for compilers than it needs to be, but I also wonder if the comment about Pascal isn't presuming the same degree of optimizations.

It's also worth considering that how you use a language has a lot to do with compile times. I once had a template-heavy C++ file that took a couple minutes to compile and doing so used a couple GB of memory. Once I eliminated some unnecessary template parameter type deduction, it took only a few seconds to compile and I think memory usage dropped accordingly. Although this particularly bad example involves C++, I can imagine things one might do in C that also create needless burden, such as having lots of overlong and inline functions.

Part of the software engineering discipline is understanding how to use programming language features in a scalable and maintainable way.

[CochainComplex]

Is a Ferrari really the most appropriate image to represent faster build times? I'm just saying that it can take a year or two from order to delivery when you buy a Ferrari.

A Ford Model T, OTOH, would have been very appropriate.

*cough* ..as a German I'm obliged to point out that there a plenty of high quality options available. Just to name the obvious ones: Porsche, Mercedes, BMW, VW ....and then there are the acquired brands running on german engines: Lamborghini, Bugatti...but with the later ones you have issues with the shipmenttime aswell.

d796fd6d16fe14ec6912213bf12c7829.jpg....;P

[coder]

Problem is, Phoronix is a very technical site, and for technical articles it is difficult to associate images with them, especially when the concepts are so abstract that they don't exist or cannot be represented with an object.

I'd probably use images from a manufacturing/assembly-line as the motif for doing builds.

If you wanted more abstract iconography, I'd go with some kind of hand-crank grinder with streams of C statements flowing into it and chunks of binary being spit out the other side.

[milkylainen]

It should mostly be a matter of including new header files. The function signatures and types aren't changing, just the location of their definitions. It's nothing that those who don't mainline their modules don't already deal with.

Agreed, but to what extent? Is everything moved about?
Or is it like 99% is going to be the same but a few headers that needed more work?

Oh well. I haven't looked into the changeset, so might aswell do that.
Thanks.