Zstd-Compressed Linux Kernel Images Look Very Close To Mainline With Great Results

The work on Zstd'ing the Linux kernel for using this Facebook-developed Zstandard compression algorithm to in turn speed up decompression times when booting Linux kernel images might be mainlined as soon as Linux 5.9.

Nick Terrell of Facebook on Thursday night sent out the eighth version of these Zstd patches for allowing the Linux kernel image to be compressed with it. Those patches were sent out again as a pull request though there does appear to be some rather trivial code tweaks to make at this point.

The patches show great potential for using Zstd to compress the kernel, ramdisk, and initramfs.

Facebook engineers found that on x86_64 hardware when switching from XZ to Zstd for the initial RAM file-system that the decompression time dropped from 12 seconds to just 3 seconds. This XZ to Zstd switch yielded overall a two second boot time improvement. Also seeing a large improvement is if going from LZMA to Zstd where Facebook found on those servers the decompression time went from 27 to 8 seconds.

As for the compression ration, Zstd is smaller than the Gzip'ed kernel but larger than XZ or LZMA compressed images. Zstd though is the fastest at decompression with the exception of LZ4.

It's looking good that this Zstd support will hopefully make it into the next kernel release (Linux 5.9) otherwise the follow on if timing doesn't work out quite right.

Once mainlined, a follow up plan is to drop BZ2 and LZMA(1) support from the kernel. This in turn will allow cleaning up the kernel code a bit more as Bzip2 and LZMA are not used anywhere else currently within the kernel tree. Adam Borowski commented, "If you draw a speed-vs-size graph, at no point bzip2 or lzma are a good choice, while zstd wins by a large margin for most of the range."