Ubuntu 21.10 Compressing Debian Packages With Zstd

[phoronix]

Phoronix: Ubuntu 21.10 Compressing Debian Packages With Zstd

It's coming three years later than originally planned but with Ubuntu 21.10 this autumn the Debian packages will now be compressed via Zstd for offering speedier decompression speeds...

https://www.phoronix.com/scan.php?page=news_item&px=Ubuntu-21.10-Zstd-Debs

[shmerl]

Time for Debian to do it as well. Faster decompression times will be great.

[middy]

arch switched over to zstd back in december of 2019 for packages and for initramfs this year. really sped up decompression.

[skeevy420]

I wonder if there'll ever be a dynamic Zstd dictionary framework for compressing packages based on the package's files + its sources or even a massive dictionary based on every package in the OS and its sources? The former could be something created per package that gets better each time the package updates and gets compressed again (like per package ccache) while the latter could be a project shared among the community as the Zstd Linux Packaging Dictionary.

I wonder how long it would take to index every Debian package and source file...and then repeat that process with Red Hat, Fedora, SUSE, Arch, Ubuntu, etc...and if that would even matter in regards to OS packaging efficiency.

[jacob]

Are there any benchmarks for this? Intuitively I would have thought that decompression times would be insignificant compared to download times (and the actual disk writes) and a slower but higher ratio compressor would thus lead to better performance overall?

[caligula]

"Facebook-developed Zstandard compression technology"

I'm pretty sure zstd was invented long before the developer was hired by FB.

[lyamc]

"Facebook-developed Zstandard compression technology"

I'm pretty sure zstd was invented long before the developer was hired by FB.

Hence why he used the word "developed" and not "invented"

[yoshi314]

there are quite a few people who really hate zstd because company behind its implementation. same as people who shun selinux for the very same reason.

[arun54321]

Are there any benchmarks for this? Intuitively I would have thought that decompression times would be insignificant compared to download times (and the actual disk writes) and a slower but higher ratio compressor would thus lead to better performance overall?

You search for benchmarks on arch forums.

[cl333r]

there are quite a few people who really hate zstd because company behind its implementation. same as people who shun selinux for the very same reason.

As long as it's truly open source I don't mind much.
What I really like about zstd is that it finally transitions the Linux stack to use a modern/easy API rather than the old convoluted mess that is zlib.

Zstandard API is designed with learning curve in mind. At the top, you'll find simple methods, using trivial arguments and behavior. Then, at each new paragraph, the API introduces new concepts and parameters, giving gradually more control for advanced usages. [1]

[1] https://facebook.github.io/zstd/