Ubuntu 23.10 Is Maxing Out Zstd Compression For Its Kernel Build

[phoronix]

Phoronix: Ubuntu 23.10 Is Maxing Out Zstd Compression For Its Kernel Build

Dimitri John Ledkov of the Ubuntu kernel team has written about some of the improvements made for the default kernel build on Ubuntu 23.10. Ubuntu's Linux kernel build is now using much less disk space, lower RAM use, and much faster initrd generation...

https://www.phoronix.com/news/Ubuntu-23.10-Kernel-Build

[bachchain]

How does more compression translate to faster image generation and lower RAM usage?

[Malsabku]

These kind of optimisations are the reason why Ubuntu is better for the desktop than Debian.

[reba]

These kind of optimisations are the reason why Ubuntu is better for the desktop than Debian.

Because it impacts you how?

And is that tradeoff (Canonical/Microsoft politics, nonfree snap lock-in, gtk-isms) worth it?

[billyswong]

How does more compression translate to faster image generation and lower RAM usage?

Below is my guess:
The "assembling the initrd using split CPIO archives" part means the compression and extraction process is split into smaller pieces, each with a smaller dictionary or something like that. Thus the lower RAM usage requirement in boot time.

For faster image generation, it is likely because now they aren't recompressing everything from scratch but making substantial part of the kernel image generation a process of concatenation of precompressed modules.

So essentially we are comparing a solid archive with lower compression setting, and a non-solid archive with higher compression setting.

[satadru]

Hopefully, they will work on getting the changes they have made upstreamed... It doesn't look like these are large patches...

[peterdk]

So, from the blog I understand that the download size is actually increased. I didn't understand why the numbers were reversed, but it seems that the download size is now 949MB vs 600MB. Still don't get why that is the case.

[jabl]

How does more compression translate to faster image generation and lower RAM usage?

So AFAIU how it worked earlier: kernel module and firmware packages were compressed with whatever is nowadays the default .deb package compression method (Zstd, or is it still bzip2?). When installing, the module files and firmware files were installed uncompressed into the filesystem. Then when you install a new kernel, it will generate an initrd for that kernel by taking a subset of the modules and firmware files, and then compress the initrd with zstd.

With the new system, the individual modules and firmware files are compressed with zstd, relying on the in-kernel zstd implementation to uncompress them when they're loaded. As the individual files are already compressed, the .deb packages themselves are uncompressed. And when creating the initrd, it can just concatenate the necessary module and firmware files without uncompressing and then recompressing the final initrd image. So the benefits are 1) Less disk space usage as the modules and firmware image files are stored compressed on the filesystem 2) Less unnecessary uncompressing and recompressing the same data. Only the data that is actually used will be uncompressed at load time.

[guglovich]

I was waiting for the line that says it's not a credit to Ubuntu. It's business as usual.

[Malsabku]

Because it impacts you how?

And is that tradeoff (Canonical/Microsoft politics, nonfree snap lock-in, gtk-isms) worth it?

I'm just interested in the best end-user-experience. All your points are ideological.

If you want to know what a real lock-in is, you should take a look at your phone. And that's where you should start your criticism.