Updated Zstd Planned For Linux 5.16 With Better Performance

As reported on last week, an updated Zstd implementation for the Linux kernel is being re-attempted by Zstd developer Nick Terrell at Facebook. Today he sent out the latest Zstd kernel patches to provide a much newer version of the code compared to what is currently mainlined and will provide much better performance and numerous fixes.

The Zstd code currently within the Linux kernel is out-of-date and it's taken an unfortunate amount of time to get it updated. Fortunately, the new code is introducing a new kernels-style wrapper API around Zstd that should allow for these code updates to be performed smoother and more easily moving forward. In fact, the Zstd kernel code is working towards being automatically generated/derived from the upstream Zstd sources.

The Zstd code currently within the kernel is four years old (Zstd 1.3.1) and since then there have been many fixes and performance improvements. With this Zstd update, Btrfs Zstd compression can be a few percent faster, Btrfs Zstd decompression can be around ~15% faster, SquashFS Zstd decompression is around 15% faster too, F2FS Zstd decompression during reads can be around 20% faster, zRAM decompression can be around 30% faster, kernel Zstd image decompression can be around 35% faster, and initramfs Zstd decompression and building was around 5% faster.

Already these patches provide big speed-ups for the Linux kernel Zstd performance especially on the decompression side while Zstd 1.5.1 will be released soon with even more performance optimizations. Additionally, for logistics this patch series takes the code up to Zstd 1.4.10 while the follow-on patch once upstreamed will take it to that upcoming Zstd 1.5.1 state.

More details on this proposed Zstd updated implementation for the Linux kernel via this patch series. The hope is now to get that code pulled into Linux-Next so it can be tested more broadly over the next few weeks and then to ideally see this updated code merged next month for the Linux 5.16 merge window.