Announcement

Collapse
No announcement yet.

Btrfs LZO Compression Performance

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • drag
    replied
    Originally posted by BenderRodriguez View Post
    You may be right.
    What would matter then is the speed of your CPU. Also matters heavily the stuff your compressing.

    On my system I created a *tar file of pdf files. Some of the pdf files are text, but most of them are mainly images. Since they are already compressed quite a bit then you don't benefit from it a whole lot.

    This compresses at about 60-70 MB/s and decompresses at close to 400MB/s. The size saved is small, however. I am only saving 10-20 MB in 246MB. So using compression on something like that is not worth it.

    Meanwhile I used dd it create a 573MB file full of zeros. That compresses down to just over 2.3MB and takes 3/4 of a second.

    On the other end of a spectrum a 573MB file made using '/dev/urandom' takes almost 10 seconds to compress and is actually slightly larger afterwards.

    So even on very fast SSDs it MAY be worth it. If you care more about read speeds it may help. If you care about random access it may hurt.

    It also heavily depends on how smart the system is about using compression.


    It's stupid to compress jpg files or gif files or any sort of common multimedia files. They are already compressed heavily using specialized algorithms and it's extremely unlikely that lzo is going to help things any. But text files, documentation, program files, and many other can benefit from the compression. Some databases may benefit also, but you would think that if they did they would already be using compression internally.

    Leave a comment:


  • BenderRodriguez
    replied
    Originally posted by nbecker View Post
    I was thinking that for slow mechanical disk, lzo helps performance because compress/decompress is faster than read/write extra blocks to/from disk. But SSD maybe not.
    You may be right.

    Leave a comment:


  • nbecker
    replied
    Originally posted by BenderRodriguez View Post
    Depends on what type of files you got and how many of them. Binary/video/audio/pdf/other already compressed files don't compress too well so LZO won't help much, it will help only with those that compress well. Also as the files will take a little bit less of space it will help and lower the writes number on SSD

    The threaded writes are slower probably because it is CPU bound, without zlib/lzo the cpu only had to process (or not at all, DMA) the write while with zlib/lzo it has to compress them.
    I was thinking that for slow mechanical disk, lzo helps performance because compress/decompress is faster than read/write extra blocks to/from disk. But SSD maybe not.

    Leave a comment:


  • BenderRodriguez
    replied
    Depends on what type of files you got and how many of them. Binary/video/audio/pdf/other already compressed files don't compress too well so LZO won't help much, it will help only with those that compress well. Also as the files will take a little bit less of space it will help and lower the writes number on SSD

    The threaded writes are slower probably because it is CPU bound, without zlib/lzo the cpu only had to process (or not at all, DMA) the write while with zlib/lzo it has to compress them.

    Leave a comment:


  • nbecker
    replied
    lzo + ssd?

    I wonder if compression helps or hurts performance on ssd? Access is already fast.

    Leave a comment:


  • loonyphoenix
    replied
    Originally posted by BenderRodriguez View Post
    Why do i get the feeling that zlib/lzo mode speeds up iozone and fs-mark only because the created files are empty and thus compress almost infintely good ?
    I get that impression, too. This makes those tests completely irrelevant.

    Leave a comment:


  • FireBurn
    replied
    I know seeing these benchmarks takes me back to my reiser4 days

    I guess it really was years ahead if it's time, it's a shame that the same level or development wasn't maintained after Hans arrest

    Leave a comment:


  • ayumu
    replied
    Originally posted by Kirurgs View Post
    This bench cries to be compared to reiser4 compression abilities If you could do that it would be awesome...

    P.S. At the time of writing there is no 2.6.38 reiser4 patch yet, but it should follow shortly
    Indeed, more so considering reiser4 has had LZO compression support for many years already.

    Leave a comment:


  • BenderRodriguez
    replied
    Why do i get the feeling that zlib/lzo mode speeds up iozone and fs-mark only because the created files are empty and thus compress almost infintely good ?

    Leave a comment:


  • Kirurgs
    replied
    This bench cries to be compared to reiser4 compression abilities If you could do that it would be awesome...

    P.S. At the time of writing there is no 2.6.38 reiser4 patch yet, but it should follow shortly

    Leave a comment:

Working...
X