Zstd Compressed Linux Kernel Images Proposed Once More

[Buntolo]

archlinux already switched to ztsd for its packages in January: https://www.archlinux.org/news/now-u...e-compression/

[skeevy420]

archlinux already switched to ztsd for its packages in January: https://www.archlinux.org/news/now-u...e-compression/

And I ran it for a few months with these Zstd initramfs patches and....honestly....can't tell a difference between it and LZ4 from an end user's perspective.

The difference between Zstd or LZ4 and XZ is night and day. XZ is frickin slow compared to the other two and, for the kernel image, there is no tangible benefit trying to get an extra .003 mb with XZ when either of the other compressors boot that much noticeably faster.

Basically, if y'all haven't switched to LZ4 or Zstd kernel images, you need to get off of Phoronix right now, look up your distribution's instructions, and make the switch.

[ermo]

My custom-configured LZ4 kernels are 8-8.5 MiB apiece and I currently keep 6 of them around from different major series on a 512 MiB /boot partition (currently using 72 MiB all in all).

Outside of embedded systems with BoM-dictated constraints, I'm not really sure why distros choose to NOT use ZSTD (or even LZ4), particularly with large, generic distro kernels? Do kernel updates really make up a large enough proportion of the bandwidth used to serve users to warrant them being xz compressed?

Particularly for laptops, I would imagine that the benefits from going from xz to zstd or lz4 on boot speed would be noticeable (on the order of 2-3 seconds vs. instant perhaps).

Anecdotally, switching from xz to lz4 on an old C2Q system of mine certainly made a difference in decompression time.

[caligula]

But isn't LZ4 still slightly faster than ZSTD at decompression?

On slow ARM systems you have slow drives so the better compression ratio of zstd/xz will provide better performance overall.

[Artemis3]

archlinux already switched to ztsd for its packages in January: https://www.archlinux.org/news/now-u...e-compression/

This is a mistake. A kernel you may want decompressed quick at boot, but not install packages. For install packages, a smaller size is more important (bandwidth saving) than decompressing speed. You are not decompressing packages ALL THE TIME like you would when using btrfs compression. While Arch is rolling, and people frequently download packages, it does not take any significant amount of time to decompress and install afterwards, especially if you are keeping your system updated.

[AndyChow]

This is a mistake. A kernel you may want decompressed quick at boot, but not install packages. For install packages, a smaller size is more important (bandwidth saving) than decompressing speed. You are not decompressing packages ALL THE TIME like you would when using btrfs compression. While Arch is rolling, and people frequently download packages, it does not take any significant amount of time to decompress and install afterwards, especially if you are keeping your system updated.

This is not a mistake. The 0.8% increase in size is well worth the 1300% decompression speed, even if you only use it once. The time required for decompressing the package is reduced by more than the increase in time that downloading that package requires, for most modern connections.

[terrelln]

This is not a mistake. The 0.8% increase in size is well worth the 1300% decompression speed, even if you only use it once. The time required for decompressing the package is reduced by more than the increase in time that downloading that package requires, for most modern connections.

Additionally, because multithreading produces deterministic outputs, packagers can use multithreaded compression. That drastically speeds up compression time for large packages.

[Buntolo]

This is a mistake. A kernel you may want decompressed quick at boot, but not install packages. For install packages, a smaller size is more important (bandwidth saving) than decompressing speed. You are not decompressing packages ALL THE TIME like you would when using btrfs compression. While Arch is rolling, and people frequently download packages, it does not take any significant amount of time to decompress and install afterwards, especially if you are keeping your system updated.

While I understand your point, I've noticed drastically quicker installing times.