Fast Kernel Headers Work Restarted For Linux To Ultimately Speed Up Build Times

[V1tol]

Yes, if you configure your kernel with make allnoconfig followed by time make -j$(nproc), for which the system installing your kernel would be completely useless. I'm assuming on a stock distribution config (make allmodconfig gets you pretty close to this), you're going to clock somewhere between 10-15 minutes, and packaging overhead to make your build redistributable would add another 3 or so minutes.

My build using ccache and nearly stock Arch config takes somewhere between 5 to 10 minutes on 5950X depending on diff size. Clean build takes somewhere around 25 minutes. That is including O3 patches and Clang LTO by default. Didn't want to strip unneeded modules - too much hassle and I can go drink coffee, touch grass or watch Youtube while it is compiling.

[bachchain]

Has every single other problem or feature been completed and now they can focus on compiling it even faster than under 30 seconds?

Seriously, what is it with this mentality?

Dude, it's a kernel. Until the day people stop making new hardware, it will never be "complete"

[Doomer]

But let's be honest with ourselves, there's no better place to spend their energy than how fast the kernel can be compiled?

such as?

I say it's fast enough, concentrate on other areas,

or concentrate on both? there is enough man power.

Even better, how about they restructure the Linux kernel so that drivers aren't loaded modules of the kernel, you simply install drivers the way you do on Windows.

feel free to submit to linus gang the technicalities behind your idea, or even better, show us a proof of concept.

It's time to move past Linux being a half-assed design.

can you provide literature on this subject or it's just your uneducated opinion on the matter?

[FedFer]

sophisticles making their smart comments again, what a great day /s

[flakmirror]

Well, for people about complaining this, I can imagine some integration/regression tests can make use of this, setting up a CI pipeline or such. Doesn't matter for me, but if it makes the kernel build any easier/faster (and won't affect anything else), what's really the issue?

[sophisticles]

So that's how the other side lives! It takes my laptop around 90 minutes to run a fresh make bindeb-pkg!

Let me cheer you up, my 4C/8T Ice Lake laptop that i always run with the CPU governor set to powersave and never gets above 2Ghz took 133 minutes, 49 seconds to compile the kernel with allmodconfig.

[sophisticles]

.feel free to submit to linus gang the technicalities behind your idea, or even better, show us a proof of concept.

can you provide literature on this subject or it's just your uneducated opinion on the matter?

You're right, I'm just a silly, silly man, with not much education, you will have to excuse me, i only went to a public college.

I wonder if I can find any literature from a trusted source, maybe from someone that the Linux community and Linus and associates can trust:

https://www.kernel.org/doc/html/v5.11/admin-guide/README.html

"make tinyconfig" Configure the tiniest possible kernel.

"make allmodconfig" Create a ./.config file by setting symbol values to 'm' as much as possible.

• NOTES on make config:
  • Having unnecessary drivers will make the kernel bigger, and can under some circumstances lead to problems: probing for a nonexistent controller card may confuse your other controllers.
  • A kernel with math-emulation compiled in will still use the coprocessor if one is present: the math emulation will just never get used in that case. The kernel will be slightly larger, but will work on different machines regardless of whether they have a math coprocessor or not.
  • The “kernel hacking” configuration details usually result in a bigger or slower kernel (or both), and can even make the kernel less stable by configuring some routines to actively try to break bad code to find kernel problems (kmalloc()). Thus you should probably answer ‘n’ to the questions for “development”, “experimental”, or “debugging” features.

Bottom line is that no one should ever do "make allmodconfig", they should do "make tinyconfig" and then have a desktop app that allows the end user to install the relevant modules for their hardware,

To give you an idea of the speed difference, my 4C/8T Ice Lake using the powersave governor and under 2Ghz, with dozens of Waterfox tabs open and a Youtube vidoe running, took the following times to compile the kernel:

make allmodconfig, 133 minutes, 49 seconds

make default, 11 minutes, 4 seconds

make tinyconfig, 47 seconds.

So you can either spend tens of thousands of dollars upgrading your hardware, spends thousands of man hours trying to improve compilation times via various methods or you can use your brain and use what already exists and simply change your workflow slightly.

Your choice.

[toves]

Even better, how about they restructure the Linux kernel so that drivers aren't loaded modules of the kernel, you simply install drivers the way you do on Windows.

This way the kernel is very small and fast and you don't have to worry about compiling support for every Godforsaken device in existence.

I am not sure that Windows does things all that differently other than probably having a well defined kernel / device driver interface. Windows might load device driver code into separate memory segments rather than into kernel memory but I suspect not. I imagine you would get similar results from user space device drivers which apart from potential performance limitations could implement much of your suggestion in much the same way user space file systems do. In the wash up its still a process of run time linking which is basically gluing names and services together, however its done.

A linux kernel only needs to be built to support kernel modules whether you build the modules themselves or which modules you do build is optional.
I imagine kernel developers built the whole shebang to shake out potential dependency and other problems.
I know from years ago trying to port an unsupported scsi adaptor to linux running on a Dec alpha that building a minimal kernel and then working on the driver did speed things up (mostly reboot times )

Personally I don't worry too much about people pursuing what are at face value futile goals or projects - whatever melts their butter. There are, obviously depending on your own perspective, an unbelievable number of them so apart from keeping idle hands from the devil's employment these project must provide something positive to their participants.
Linux itself was originally one such project

[Sonadow]

Yes, if you configure your kernel with make allnoconfig followed by time make -j$(nproc), for which the system installing your kernel would be completely useless. I'm assuming on a stock distribution config (make allmodconfig gets you pretty close to this), you're going to clock somewhere between 10-15 minutes, and packaging overhead to make your build redistributable would add another 3 or so minutes.

A stock distribution config built on a Supermicro workstation with two E5 2690-v2 processors entirely on a 80GB ramdisk takes almost 30 minutes.

That same stock distribution config built on an i7 Skylake notebook takes almost two hours.

That same stock distribution config built on a Gemini Lake notebook takes almost four hours.

My kconfig requires 60 - 90 minutes on the Supermicro workstation, four hours-ish on the Skylake notebook, and an entire working day on the Gemini Lake.

Unless this work is going to make these build times drop from 90 minutes to 20 minutes / four hours to one hour / eight hours to two hours-ish, they are of little relevance.

[Sonadow]

Bottom line is that no one should ever do "make allmodconfig", they should do "make tinyconfig" and then have a desktop app that allows the end user to install the relevant modules for their hardware

And for that to happen, a stable unchanging kernel ABI is required so that a driver package for kernel version X can still be installed on kernel version X,Y or even kernel version X+1, and that hardware vendors don't have to waste time building modules against every single damn version of the kernel.

But we all know why that has never happened for Linux.

Also, how is it possible to build a allmodconfig in only 133 minutes when it takes a Supermicro workstation with two E5 2690-v2 processors 60 - 90 minutes to build my kconfig, even when the whole build is done entirely on a 80GB ramdisk. Any secret sauce?