Fwupd Switches From XZ To Zstd Compression: More Trust & Slightly Better Performance

[phoronix]

Phoronix: Fwupd Switches From XZ To Zstd Compression: More Trust & Slightly Better Performance

Driven by the XZ security fiasco with malicious code aimed at remote code execution, more open-source projects are re-evaluating their dependence on XZ out of an abundance of caution. The latest to take action is the Fwupd Linux firmware updating utility with LVFS that will now prefer Zstd compression instead of XZ...

https://www.phoronix.com/news/Fwupd-Switches-To-Zstd

[arekm]

Interesting. From my observation zstd compressed worse than xz (both on some "max"ish compression settings) and I was compressing text logs. Did that improve recently in zstd?

Edit: short answer is "no" and before adding another answer please read entire thread.

[varikonniemi]

xz should be abandoned. Upstream malaiciousness alone is kind of unforgivable, but from what i have read the format itself is poorly and overly engineered.

[Robust0522]

Interesting. From my observation zstd compressed worse than xz (both on some "max"ish compression settings) and I was compressing text logs. Did that improve recently in zstd?

When Arch switched to Zst some time ago, they reported a 0.8% increase in package size but a 1300% increase in compression performance.

[Anux]

Interesting. From my observation zstd compressed worse than xz (both on some "max"ish compression settings) and I was compressing text logs. Did that improve recently in zstd?

FWUPD most likely didn't use max/extreme compression, because the time and resource usage with xz would be ridiculous. And zstd can easily compress faster at same compression ratio for xz -5.

[dralley]

Interesting. From my observation zstd compressed worse than xz (both on some "max"ish compression settings) and I was compressing text logs. Did that improve recently in zstd?

It depends on what you're compressing. Rarely is it possible to make accurate blanket statements about something like this.

Zstd is usually a little worse, sometimes a little better in my experience. Either way, it's not by much, and the performance considerations generally make zstd the better choice even if xz compresses a little better.

[Robust0522]

It's a sad story. Lasse Collin developed and maintained Xz as an open source compression tool for well over a decade until mental health issues set in. His reward is to be remembered as the guy who allowed a malicious actor to introduce a backdoor into the software which could easily not have been caught. Not a great advertisement for being an open source maintainer.

[roughl]

Interesting. From my observation zstd compressed worse than xz (both on some "max"ish compression settings) and I was compressing text logs. Did that improve recently in zstd?

Size is not everything with regard to compression. For example ArchLinux decided to switch to zstd for it's packages, which increased the package size by 0.8% but, the decompression time saw a ~1300% speedup: https://archlinux.org/news/now-using...e-compression/.

[arekm]

Size is not everything with regard to compression.

It is in my usage and that's why I asked. Edit: and the short answer is "no"

[uid313]

xz should be abandoned. Upstream malaiciousness alone is kind of unforgivable, but from what i have read the format itself is poorly and overly engineered.

It is not that the guy who wrote xz decided to be malicious. A guy started contributing to the XZ Utils project several years ago and gained trust then using sock puppet accounts pressured the unpaid, time-constrained, single maintainer into making him a maintainer.

https://arstechnica.com/security/2024/04/what-we-know-about-the-xz-utils-backdoor-that-almost-infected-the-world/

The following year, JiaT75 submitted a patch over the xz Utils mailing list, and, almost immediately, a never-before-seen participant named Jigar Kumar joined the discussion and argued that Lasse Collin, the longtime maintainer of xz Utils, hadn’t been updating the software often or fast enough. Kumar, with the support of Dennis Ens and several other people who had never had a presence on the list, pressured Collin to bring on an additional developer to maintain the project.