Fast Kernel Headers v2 Posted - Speeds Up Clang-Built Linux Kernel Build By ~88%

[zboszor]

So, the latter

I know, I have read mingo's mail.

[bbrigg]

As a part time Linux user I can appreciate the time and effort put in to streamline the kernel build process and the enormous reduction in build time is very worthwhile for those who rebuild the kernel often.

Most casual Linux users may never build the kernel, instead just applying the distribution updates. While I have built kernels whenever I needed support for new features not in the distribution channel yet and sometimes just to have a kernel built specifically for my system without including unnecessary options that I don't have, but I'm pretty sure that I'm in the minority.

Even so, all users will see benefits if those who are contributing to the kernel can spend more time on improvements to kernel performance and faster implementation of new hardware features. Anything that helps the developers to streamline their work process benefits all Linux users.

[bachchain]

Do some math. It's about cutting build time by 88% now with CLANG according to Michael .
100% / (100% - 88%) = 100 / 12 = 8.3333 , i.e. over 8x speedup already.

Unless Michael misinterpreted the numbers and it's only really speeding up by 88%, which is a totally different metric which means cutting build time by about 47%:
1 / 1.88 = 0.5319

#
# Performance counter stats for 'make -j96 vmlinux' (3 runs):
#
# (Elapsed time in seconds):
#
v5.16-rc7:
231.34 +- 0.60 secs, 15.5 builds/hour # [ vanilla baseline ]
-fast-headers-v1:
129.97 +- 0.51 secs, 27.7 builds/hour # +78.0% improvement

So, the latter

[zboszor]

Wow, feels like the v4 of the patch will enable an over 2x compile time speedup.

Do some math. It's about cutting build time by 88% now with CLANG according to Michael .
100% / (100% - 88%) = 100 / 12 = 8.3333 , i.e. over 8x speedup already.

Unless Michael misinterpreted the numbers and it's only really speeding up by 88%, which is a totally different metric which means cutting build time by about 47%:
1 / 1.88 = 0.5319

[tiennou]

Are you planning already some benchmark comparison between the original kernel and the fast header version ? Would be nice to confirm it doesn't generate any slowdown a runtime.

[uxmkt]

Does anyone have this patch available in a single file by chance?

Uh, just generate it yourself the way you need it?

git clone git://git.kernel.org/pub/scm/linux/kernel/git/mingo/tip.git;
cd tip; git diff c9e6606c7fe9..391ce485ced0

[Mark Rose]

Does anyone have this patch available in a single file by chance?

No. It's something like 100k lines. You can check out the git repository linked in the post and render the 2000+ commits to a single diff though, I imagine.

[rmfx]

Wow, feels like the v4 of the patch will enable an over 2x compile time speedup. When other devs start contributing, there will be a before and an after.

[Mark Rose]

I wonder if these header changes are going to break every external kernel module build there is?
Probably a whole lot of other stuff that are halfway between userspace and whatnot.

Does anyone know how these changes interact with existing code?
Ie, how much breakage can one expect?

It should mostly be a matter of including new header files. The function signatures and types aren't changing, just the location of their definitions. It's nothing that those who don't mainline their modules don't already deal with.

[bug77]

Holy crap. Now C isn't the fastest language to compile. Imagine if Linux was written in Pascal. Would save millions in server hardware.

Off topic, but I'm constantly amazed to keep even experienced developers that fail to grasp two simple things (imho):
1. the more featured a language is, the more it takes to compile; you want generics, type inference, even inheritance, named parameters, be prepared to pay a price
2. longer compile times are always justified when they result in faster production code (which is almost all the time); languages like Java or C# may appear fast to compile, but they do not link anything - you'll pay a hefty runtime penalty for that

More on topic, the biggest expectation I have from these patches, whenever they manage to land, is that shorter build times will make fuzzing more feasible, so a lot more bugs will be exposed and patched.