Fast Kernel Headers v2 Posted - Speeds Up Clang-Built Linux Kernel Build By ~88%

[bbrigg]

As a part time Linux user I can appreciate the time and effort put in to streamline the kernel build process and the enormous reduction in build time is very worthwhile for those who rebuild the kernel often.

Most casual Linux users may never build the kernel, instead just applying the distribution updates. While I have built kernels whenever I needed support for new features not in the distribution channel yet and sometimes just to have a kernel built specifically for my system without including unnecessary options that I don't have, but I'm pretty sure that I'm in the minority.

Even so, all users will see benefits if those who are contributing to the kernel can spend more time on improvements to kernel performance and faster implementation of new hardware features. Anything that helps the developers to streamline their work process benefits all Linux users.

[zboszor]

So, the latter

I know, I have read mingo's mail.

[milkylainen]

It should mostly be a matter of including new header files. The function signatures and types aren't changing, just the location of their definitions. It's nothing that those who don't mainline their modules don't already deal with.

Agreed, but to what extent? Is everything moved about?
Or is it like 99% is going to be the same but a few headers that needed more work?

Oh well. I haven't looked into the changeset, so might aswell do that.
Thanks.

[coder]

Problem is, Phoronix is a very technical site, and for technical articles it is difficult to associate images with them, especially when the concepts are so abstract that they don't exist or cannot be represented with an object.

I'd probably use images from a manufacturing/assembly-line as the motif for doing builds.

If you wanted more abstract iconography, I'd go with some kind of hand-crank grinder with streams of C statements flowing into it and chunks of binary being spit out the other side.

[CochainComplex]

Is a Ferrari really the most appropriate image to represent faster build times? I'm just saying that it can take a year or two from order to delivery when you buy a Ferrari.

A Ford Model T, OTOH, would have been very appropriate.

*cough* ..as a German I'm obliged to point out that there a plenty of high quality options available. Just to name the obvious ones: Porsche, Mercedes, BMW, VW ....and then there are the acquired brands running on german engines: Lamborghini, Bugatti...but with the later ones you have issues with the shipmenttime aswell.

d796fd6d16fe14ec6912213bf12c7829.jpg....;P

[coder]

Off topic, but I'm constantly amazed to keep even experienced developers that fail to grasp two simple things (imho):
1. the more featured a language is, the more it takes to compile; you want generics, type inference, even inheritance, named parameters, be prepared to pay a price
2. longer compile times are always justified when they result in faster production code (which is almost all the time); languages like Java or C# may appear fast to compile, but they do not link anything - you'll pay a hefty runtime penalty for that

More on topic, the biggest expectation I have from these patches, whenever they manage to land, is that shorter build times will make fuzzing more feasible, so a lot more bugs will be exposed and patched.

That's a bit simplistic. I'm not sure there aren't a few things in C that needlessly make work harder for compilers than it needs to be, but I also wonder if the comment about Pascal isn't presuming the same degree of optimizations.

It's also worth considering that how you use a language has a lot to do with compile times. I once had a template-heavy C++ file that took a couple minutes to compile and doing so used a couple GB of memory. Once I eliminated some unnecessary template parameter type deduction, it took only a few seconds to compile and I think memory usage dropped accordingly. Although this particularly bad example involves C++, I can imagine things one might do in C that also create needless burden, such as having lots of overlong and inline functions.

Part of the software engineering discipline is understanding how to use programming language features in a scalable and maintainable way.

[coder]

can we get 1 bargraph ?

Exactly.

I just click with the hope that the article would actually have at least some build benchmarks, if not also a few runtime tests.

[perpetually high]

Just gave the patchset a whirl. Freakin incredible.

Cut my kernel build time on the 5800X from roughly 7min 10s seconds to 4min 40s

Ingo, you brilliant son of a gun! That's seriously an incredible effort that is going to be pay huge dividends over time.

Here's the git diff I used to make the patch if anyone needs:

$ git diff 391ce485ced0e47bf8d2ce8bc32bb87887e16656 c9e6606c7fe92b50a02ce51dda82586ebdf99b48 > fast_kernel_headers-v2.patch
$ patch -p1 < ./fast_kernel_headers-v2.patch

Should see CONFIG_FAST_HEADERS already enabled in the config. It did fail on "nfsd" which Is NFS server support, so I disabled that from module to disabled in the config since I'm not using it. I'm also getting a failure on linux/device_api_lock.h not being found when compiling the linux-tools package, usbip in particular. Not sure what's going on there, but this is exciting stuff.

[oleid]

Are you planning already some benchmark comparison between the original kernel and the fast header version ? Would be nice to confirm it doesn't generaWhy would runtime speed change? This is Scott shifting code in headsets.
te any slowdown a runtime.

Why would it change runtime speed? This is about changing the include hierarchy. There is zero chance it will have a runtime impact.

edit: we did something similar a few years ago, albeit with a c++ codebase: Find out what headers get included all the time and slim them down (by including less and replacing actual class definitions with forward declarations; the definitions are most of the time only needed in the cpp file).

[sinepgib]

Why would it change runtime speed? This is about changing the include hierarchy. There is zero chance it will have a runtime impact.

Because it also changed some functions to not be suggested for inlining. Depending on whether the compiler decided to follow the hint and whether that hint was clever or not, that could make runtime performance slightly better or slightly worse. I doubt the change would be huge tho.