GCC 10 Lands The Zstd LTO Bytecode Compression Support

[phoronix]

Phoronix: GCC 10 Lands The Zstd LTO Bytecode Compression Support

As a follow-up to the story about GCC eyeing Zstd compression for its LTO data, that support has now been merged into the GCC 10 code compiler...

http://www.phoronix.com/scan.php?page=news_item&px=GCC-10-Gets-Zstd-LTO-Byte-Comp

[pkese]

Just a note: While zstd and LZ4 are roughly equivalent in speed and compression ratio (in comparison to zlib and xz), zstd has a catchier name and therefore will probably achieve higher market share than LZ4.

If you'd look at some benchmarks, you'd see that there are enough differences between these two algorithms, that there is space for each:
- LZ4 offers faster compression (if you don't care about size)
- if compression size is important, then ZSTD will generally perform better (either compress better at same compression time, or offer faster compression at same size)
- LZ4 is faster to decompress and therefore better if you need to compress once and then decompress many times,
- but if you measure whole cycle (compression and decompression) then ZSTD often turns out faster (somewhat depending on compression ratio) - I assume that this was the use-case for GCC.

I'm not sure about names, they both seem catchy to me.

[pkese]

Actually, both LZ4 and ZSTD were written by the same author, Yann Collet.
In one of Yann's blog posts, describing the algorithm, he states: [zstd's] Lower levels are mapped onto LZ4, being very fast. Upper levels can be very complex, on top of very slow, offering improved compression ratio.

[eva2000]

zstd also has negative compression levels -10 to -1 in addition to levels 1-22 - very flexible if you want to choose between speed and compression ratios

[hubicka]

In GCC use decompression speed is more important, since compression happens at compile time and decompression is done at link-time. So with compile/edit/rebuild cycle you compress one file but decompress all of them each LTO relink.