Updated Zstd Planned For Linux 5.16 With Better Performance

[FPScholten]

The Xanmod kernel has incorporated these zstd patches for quite some time now.

[krzyzowiec]

Forgive my brevity, am on mobile and little sleep. I use zstd for both zram and storage compression on a 64gb 4gb ram dualcore celeron.

lz4 while faster has lower compression ratio. almost a difference of 2.1x in lz4, and 2.8x on zstd, while it sounds small, when you need as much space as you can get, it makes a difference.

for me, zstd enabled me to once again use the device for daily usage, webbrowsing, typing and music simultaneously. as now even basic research can saturate low ram.

You should really consider running Xanmod kernel if you aren't already. It literally cut my memory usage in half, from 15GB to 7GB on my desktop, and from around 7GB down to 4GB on my laptop. It made my laptop much more usable (8GB max memory).

[arQon]

You're partially right

How gracious of you to *almost* admit to being completely wrong...

zstd it has ... MUCH faster decompression speed compared to lz4

[arQon]

Note running at IO speed with a much larger file can mean actual decompression takes longer than one that doesn't match IO speed but leads to a much smaller file.

Theoretically true, sure, but in practice almost never unless you're using absolutely bottom-tier storage like an SD card / USB 2 / etc. Even on spinning rust it's probably a wash.

> At least that's the effect on my boot time with compressing the initramfs with zstd and lz4 on a slow computer (SSD mounted on SATA-II netting about 300MiB/s, Atom N270 CPU).

I don't have anything QUITE that weak to compare against, but it seems very unlikely that a weak Atom can provide throughput of > 300MB/s on any nontrivial decompressor, even given has significant RAM use and/or per-use init overhead. (Which is a perfectly valid tradeoff to make, and very much a "good" one for initramfs specifically, given its size and the knowledge that all the RAM in the machine is currently available).

On my (quite a lot faster) Atom, initramfs creation is slower than dirt, and a constant annoyance. I'd rather just have it be uncompressed, but keep forgetting to change it. Thanks for the reminder.

[Quackdoc]

You should really consider running Xanmod kernel if you aren't already. It literally cut my memory usage in half, from 15GB to 7GB on my desktop, and from around 7GB down to 4GB on my laptop. It made my laptop much more usable (8GB max memory).

Ill give it a try sometime, Currently using zenkernel, and it seems to work well, interesting to see what it will do on my system

[geearf]

You should really consider running Xanmod kernel if you aren't already. It literally cut my memory usage in half, from 15GB to 7GB on my desktop, and from around 7GB down to 4GB on my laptop. It made my laptop much more usable (8GB max memory).

How is the patchset achieving lower memory usage? Is it about using less for cache?

[krzyzowiec]

How is the patchset achieving lower memory usage? Is it about using less for cache?

No it's not changing cache usage. My system has 64GB of memory for example, and currently I can see it is using 21GB for caching purposes with 'free -m', but the actual memory usage of running applications is 8.2GB rather than the usual 15GB. I honestly have no idea what it does differently than official or Ubuntu kernels. Have you tried installing and running it? I'm curious if you see the same benefits on your machines. (and what OS/kernel you normally run)

[geearf]

No it's not changing cache usage. My system has 64GB of memory for example, and currently I can see it is using 21GB for caching purposes with 'free -m', but the actual memory usage of running applications is 8.2GB rather than the usual 15GB. I honestly have no idea what it does differently than official or Ubuntu kernels. Have you tried installing and running it? I'm curious if you see the same benefits on your machines. (and what OS/kernel you normally run)

I have not tried it, maybe I should, but my RAM usage is quite inconsistent so without running something like PTS I wouldn't really know if it was faring better.
If not cache, maybe something to do with zswap/zram? I am really curious but can't think of anything else that could impact it.