Announcement

Collapse
No announcement yet.

Updated Zstd Planned For Linux 5.16 With Better Performance

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • geearf
    replied
    Originally posted by krzyzowiec View Post

    No it's not changing cache usage. My system has 64GB of memory for example, and currently I can see it is using 21GB for caching purposes with 'free -m', but the actual memory usage of running applications is 8.2GB rather than the usual 15GB. I honestly have no idea what it does differently than official or Ubuntu kernels. Have you tried installing and running it? I'm curious if you see the same benefits on your machines. (and what OS/kernel you normally run)
    I have not tried it, maybe I should, but my RAM usage is quite inconsistent so without running something like PTS I wouldn't really know if it was faring better.
    If not cache, maybe something to do with zswap/zram? I am really curious but can't think of anything else that could impact it.

    Leave a comment:


  • krzyzowiec
    replied
    Originally posted by geearf View Post

    How is the patchset achieving lower memory usage? Is it about using less for cache?
    No it's not changing cache usage. My system has 64GB of memory for example, and currently I can see it is using 21GB for caching purposes with 'free -m', but the actual memory usage of running applications is 8.2GB rather than the usual 15GB. I honestly have no idea what it does differently than official or Ubuntu kernels. Have you tried installing and running it? I'm curious if you see the same benefits on your machines. (and what OS/kernel you normally run)

    Leave a comment:


  • geearf
    replied
    Originally posted by krzyzowiec View Post

    You should really consider running Xanmod kernel if you aren't already. It literally cut my memory usage in half, from 15GB to 7GB on my desktop, and from around 7GB down to 4GB on my laptop. It made my laptop much more usable (8GB max memory).
    How is the patchset achieving lower memory usage? Is it about using less for cache?

    Leave a comment:


  • Quackdoc
    replied
    Originally posted by krzyzowiec View Post

    You should really consider running Xanmod kernel if you aren't already. It literally cut my memory usage in half, from 15GB to 7GB on my desktop, and from around 7GB down to 4GB on my laptop. It made my laptop much more usable (8GB max memory).
    Ill give it a try sometime, Currently using zenkernel, and it seems to work well, interesting to see what it will do on my system

    Leave a comment:


  • arQon
    replied
    Originally posted by sinepgib View Post
    Note running at IO speed with a much larger file can mean actual decompression takes longer than one that doesn't match IO speed but leads to a much smaller file.
    Theoretically true, sure, but in practice almost never unless you're using absolutely bottom-tier storage like an SD card / USB 2 / etc. Even on spinning rust it's probably a wash.

    > At least that's the effect on my boot time with compressing the initramfs with zstd and lz4 on a slow computer (SSD mounted on SATA-II netting about 300MiB/s, Atom N270 CPU).

    I don't have anything QUITE that weak to compare against, but it seems very unlikely that a weak Atom can provide throughput of > 300MB/s on any nontrivial decompressor, even given has significant RAM use and/or per-use init overhead. (Which is a perfectly valid tradeoff to make, and very much a "good" one for initramfs specifically, given its size and the knowledge that all the RAM in the machine is currently available).

    On my (quite a lot faster) Atom, initramfs creation is slower than dirt, and a constant annoyance. I'd rather just have it be uncompressed, but keep forgetting to change it. Thanks for the reminder.

    Leave a comment:


  • arQon
    replied
    Originally posted by RedEyed View Post
    You're partially right
    How gracious of you to *almost* admit to being completely wrong...

    Originally posted by RedEyed
    zstd it has ... MUCH faster decompression speed compared to lz4

    Leave a comment:


  • krzyzowiec
    replied
    Originally posted by Quackdoc View Post

    Forgive my brevity, am on mobile and little sleep. I use zstd for both zram and storage compression on a 64gb 4gb ram dualcore celeron.

    lz4 while faster has lower compression ratio. almost a difference of 2.1x in lz4, and 2.8x on zstd, while it sounds small, when you need as much space as you can get, it makes a difference.

    for me, zstd enabled me to once again use the device for daily usage, webbrowsing, typing and music simultaneously. as now even basic research can saturate low ram.
    You should really consider running Xanmod kernel if you aren't already. It literally cut my memory usage in half, from 15GB to 7GB on my desktop, and from around 7GB down to 4GB on my laptop. It made my laptop much more usable (8GB max memory).

    Leave a comment:


  • FPScholten
    replied
    The Xanmod kernel has incorporated these zstd patches for quite some time now.

    Leave a comment:


  • sinepgib
    replied
    Originally posted by arQon View Post

    Citation needed. lz4 basically runs at IO speed. zstd very much does not.
    Until your post, I've never seen anyone claim, anywhere, ever, that zstd is EVER faster than lz4, let alone "MUCH" so.
    Note running at IO speed with a much larger file can mean actual decompression takes longer than one that doesn't match IO speed but leads to a much smaller file. At least that's the effect on my boot time with compressing the initramfs with zstd and lz4 on a slow computer (SSD mounted on SATA-II netting about 300MiB/s, Atom N270 CPU). At the end of the day, cat would be fastest to decompress if we left IO time out of the equation :shrug:

    Leave a comment:


  • skeevy420
    replied
    Great news. Since there are 5.14 patches I'm hoping it'll get backported to 5.15.

    On top of this, there are significant performance improvements coming down the line in the next zstd release, and the new automated update patch generation will allow us to pull them easily.
    To me, that's the best part of this announcement. Performance improvements are great; not having to wait so long for them is better. AFAICT, the only thing BTRFS is missing over ZFS in regards to Zstd is fast support...which can equal LZ4 in speeds.

    I don't know how to do this, but I was thinking and since Zstd basically decompresses just as fast regardless of level "compress-idle:X and compress-force-idle:X" mount options would be nice. Whenever the system is idle and doing nothing it would compress stuff to level X. We could use "compress-force:2,compress-force-idle:15" so the system is nice and fast when we're doing stuff and can compress higher when we're not.

    Leave a comment:

Working...
X