Linux 6.8 Looks To Upgrade Its Zstd Code For Better Compression Performance

Back in Linux 6.2 the in-kernel Zstd compression/decompression code was updated against the Zstd 1.5 upstream state. Now for the Linux 6.8 kernel in the new year the plan is for updating to Zstd 1.5.5 that should provide better compression performance.

While sadly missing out on the recently closed Linux 6.7 merge window, Nick Terrell at Meta sent out a set of kernel patches for updating the Zstd code in the Linux kernel for the Zstd 1.5.5 upstream state. This Zstd code within the kernel is used by various file-system drivers for transparent file-system compression/decompression, compressing various kernel assets into Zstd format, etc.

Helping to motivate this Zstd update for the Linux kernel is Intel wanting the newer Zstd version in the Linux kernel as it exposes Zstd's external match provider API and in turn allows QuickAssist Technology (QAT) to accelerate the LZ match finding stage... Good news for those with QAT hardware or QAT acceleration integrated into Xeon Sapphire Rapids and upcoming Emerald Rapids processors.

In addition to the API addition for Intel's needs, the updated Zstd code within the kernel has been tested and found to deliver about a 6% reduction in write+compression times. However, there is a minor increase in time for read+decompression times. Zstd 1.5.5 itself was released back in April with some performance improvements, fixes, and other changes.

Zstd 1.5.5 for the Linux kernel can be found in patches on the mailing list with a plan to submit this work for the Linux 6.8 merge window.