Zstd 1.5.6 Released - Celebrating Google Chrome Support For Zstandard Encoding

[Old Grouch]

Been waiting for this.
If everyone serving HTTP content using gzip switched to zstd ASAP, it would save countless gigawatt hours per year.
I've been using zstd for internal services for 8 years now and other companies probably are doing it too, but this allows switching the end-user facing parts of the system too.

Hmm. Really?

Using cpu cycles to decompress gzipped data uses less power than processing uncompressed data?

Really?

Show me the calculations, please. You cannot assume a 100% roll-out of Energy Efficient Ethernet.

[Mark Rose]

Been waiting for this.
If everyone serving HTTP content using gzip switched to zstd ASAP, it would save countless gigawatt hours per year.
I've been using zstd for internal services for 8 years now and other companies probably are doing it too, but this allows switching the end-user facing parts of the system too.

About 40% of the internet uses Brotli now. Given how slow the uptake has been, I don't think we'll see a rapid shift to zstd either.

https://w3techs.com/technologies/breakdown/ce-brotlicompression/ranking

Given that Brotli compresses about 5% better, and bandwidth is money, there isn't much incentive for sites to use zstd, even if zstd uses slightly less CPU than Brotli. The authors of both comment here:

https://news.ycombinator.com/item?id=19678985

[discordian]

Given that Brotli compresses about 5% better, and bandwidth is money, there isn't much incentive for sites to use zstd, even if zstd uses slightly less CPU than Brotli. The authors of both comment here:

https://news.ycombinator.com/item?id=19678985

brotli uses a static dictionary, other than that its rather uninspiring. would be interesting to see comparisons to zstd using a similar dictionary trained on http documents.

[polarathene]

Caddy has had zstd support for quite a while, it's a fairly popular web server choice these days. Although this report would make that seem unlikely, I'd imagine that many of the websites there are either on very dated servers or the deployment is either not something the site owner has much involvement/control over or they don't care (so probably not configured for Brotli either).

Last I checked nginx offered brotli via a separate module/package? Apache I think has it available out of the box, and Caddy has both out of the box. Many web servers usually get fronted by these three or some service like Cloudflare, as these are all a better part of the deployment to delegate asset compression to.

Given that Brotli compresses about 5% better, and bandwidth is money, there isn't much incentive for sites to use zstd, even if zstd uses slightly less CPU than Brotli. The authors of both comment here:

https://news.ycombinator.com/item?id=19678985

That's very close to 5 years old discussion now. Like one of the comments from the Brotli dev that responded to a benchmark that was outdated by a year or two, you can't really use such an old reference as Brotli vs zstd would have changed a fair bit since then. I haven't followed brotli but zstd sure made quite a lot of changes over that span.

I do recall other discussions in the past where the zstd window size was a notable concern. It could outperform brotli IIRC, but had considerably larger memory requirements which could be acceptable for desktops and your modern smartphones, but for some was a concern.

I do remember the lack of trained/custom dictionary support (which would make zstd pretty competitive option), but AFAIK it can't be embedded into the archive / stream for the decompressor to use. You could use zstd via WASM instead IIRC if you had a web service/app that would make enough requests per client for that extra overhead to be worthwhile.

[evergreen]

In the latest RFC proposal from the Chrome folks, zstd actually uses less memory (up to 8MB) than brotli (up to 16MB).