Intel Updates QAT Zstd Plugin For Accelerating Zstandard Compression

[phoronix]

Phoronix: Intel Updates QAT Zstd Plugin For Accelerating Zstandard Compression

Released on Sunday night was a new version of the Intel QuickAssist Technology (QAT) Zstd plug-in for accelerating Zstandard compression with QAT-enabled adapters and modern Xeon Scalable processors sporting QAT accelerators...

https://www.phoronix.com/news/Intel-QAT-Zstd-Plugin-v0.2

[uid313]

But Zstandard is already known to very fast.

[edxposed]

It may be fast enough for regular use cases, but for some special use cases, there is only faster but not the fastest, e.g. FS transparent compression, ZRAM. zstd still struggles to reach the level of lz4 in these areas.

[skeevy420]

It may be fast enough for regular use cases, but for some special use cases, there is only faster but not the fastest, e.g. FS transparent compression, ZRAM. zstd still struggles to reach the level of lz4 in these areas.

This only supports Zstd levels 1 to 12 making QAT Zstd only better if you compress files at those levels and want to both save power and increase the speeds. Zstd-fast and LZ4 can still be faster than this with the right CPU. IMHO, QAT Zstd would be a lot cooler if it supported levels 13-18 and 19-22. 1 to 12 is rather limited.

I don't think any in-kernel Linux FS supports Zstd via QAT. OpenZFS may get support for that (and more ) with ZIA.

[Anux]

This only supports Zstd levels 1 to 12 making QAT Zstd only better if you compress files at those levels

You need to keep in mind that those levels are not the same, QAT compresses much more at those levels than standard zstd.

[skeevy420]

You need to keep in mind that those levels are not the same, QAT compresses much more at those levels than standard zstd.

Are you sure that's the case? The levels being different, not QAT's performance being better when using those levels. All the documentation I read says levels 1 to 12 are where you'll get a speed and efficiency boost working with QAT due to those being QAT's supported levels. Zstd 9 with QAT is like using Zstd 3 or 4 with a CPU (IIRC). Nothing states that anything encoded with QAT is QAT specific and has to be decoded with QAT so I have to assume that QAT levels and Zstd leves are the same in regards to what Zstd sees and how it operates when just a CPU present.

[Anux]

Are you sure that's the case? The levels being different, not QAT's performance being better when using those levels.

Well I remebered it a little wrong, zstd 4,5 is circa QAT 9 in compression ratio but still 2.5 times faster:

https://community.intel.com/t5/Blogs/Tech-Innovation/Artificial-Intelligence-AI/Intel-QuickAssist-Technology-Zstandard-Plugin-an-External/post/1509818