Blosc Compressor Adds AVX-512 Bitshuffle: 20% Faster Compression For Zen 4 X3D CPU

Blosc (c-blosc2) is a high performance compressor focused on binary data for efficient storage of large binary data-sets in-memory or on-disk and helping to speed-up memory-bound computations.

With today's c-blosc2 2.11 release, AVX-512 support has been added to its Bitshuffle filter. This is based on a back-port from the upstream Bitshuffle project. In turn the Blosc project is finding up to 20% better compression speeds when testing with an AMD Ryzen 9 7950X3D (Zen 4 with 3D V-Cache) processor.

The benchmark results for AVX-512 with c-blosc2 are very promising and a nice uplift over the existing AVX2 code path. I have done blosc CPU benchmarking and will be updating to the C-blosc2 2.11 release for seeing how well this new AVX-512 support is impacting the Intel vs. AMD race.

The c-blosc2 2.11 release also adds a package definition for Guix and has other low-level improvements to this high performance compressor library.

Downloads and more details on today's c-blosc2 2.11 release via GitHub.