Announcement

Collapse
No announcement yet.

OpenMandriva Is The Latest Linux Distribution Using Zstd To Compress Packages

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • OpenMandriva Is The Latest Linux Distribution Using Zstd To Compress Packages

    Phoronix: OpenMandriva Is The Latest Linux Distribution Using Zstd To Compress Packages

    Similar to Fedora's move last year to compress RPMs with Zstd rather than XZ for much faster decompression speeds and a better compression ratio at the highest level, OpenMandriva has now enacted a similar change...

    http://www.phoronix.com/scan.php?pag...riva-Zstd-RPMs

  • #2
    Nice move. Every package should be smaller and decompress much faster. Very good

    Comment


    • #3
      If Fedora does it, then I guess eventually it will trickle down to Red Hat.

      I wonder if Debian and Gentoo are considering Zstd.

      Comment


      • #4
        I guess that Gentoo uses whatever upstream uses to compress their sources.

        You can otherwise specify to use pbuzip to uncompress source files.

        Code:
          PORTAGE_COMPRESS="lzip"
          PORTAGE_COMPRESS_FLAGS="-9"
          PORTAGE_BZIP2_COMMAND="pbzip2"
          PORTAGE_BUNZIP2_COMMAND="pbunzip2"
          BINPKG_COMPRESS="lzip"
          BINPKG_COMPRESS_FLAGS="-9"

        Comment


        • #5
          Originally posted by Spam View Post
          I guess that Gentoo uses whatever upstream uses to compress their sources.

          You can otherwise specify to use pbuzip to uncompress source files.

          Code:
          PORTAGE_COMPRESS="lzip"
          PORTAGE_COMPRESS_FLAGS="-9"
          PORTAGE_BZIP2_COMMAND="pbzip2"
          PORTAGE_BUNZIP2_COMMAND="pbunzip2"
          BINPKG_COMPRESS="lzip"
          BINPKG_COMPRESS_FLAGS="-9"
          I am not so sure about this. I have the following note in make.conf, from some time ago (year 2014 or before):

          Code:
          #PORTAGE_BUNZIP2_COMMAND="pbunzip2"     # Slower than bzip2
          Majority of bz2 archives on the Internet aren't compressed with pbzip2. Have you tested performance of pbunzip2 on archives that aren't compressed with pbzip2?

          Comment


          • #6
            Hope OpenMandriva's website starts working again soon. It's been 'off the air' for a few days at least. I don't do Facebook, so...

            Comment


            • #7
              Originally posted by willbprog177 View Post
              Hope OpenMandriva's website starts working again soon. It's been 'off the air' for a few days at least. I don't do Facebook, so...
              Same. I block Facebook at the network level as best I can. I can appreciate some of the work that their devs have made public, but I have no wish whatsoever to interact with the commercial entity.

              ...

              zstandard is one of the things to come out of Facebook devs that I am really impressed with; even if I wished it would handle compression in such a way I don't have to tar everything up before zst'ing it, or do each file individually. I'll admit it's been a while since I pulled it down, so that may have been added, however.

              Comment


              • #8
                Originally posted by Paradigm Shifter View Post
                I wished it would handle compression in such a way I don't have to tar everything up before zst'ing it, or do each file individually. I'll admit it's been a while since I pulled it down, so that may have been added, however.
                zstd is just a compression format, like gzip or xz, it still only processes one stream at a time. You still need an archive format like tar to handle multiple files and their metadata (filenames, owners/permissions, ...).

                Tar usually puts the archive _under_ the compression (which means multiple similar files compress well). 7-zip puts the archive _over_ the compression (which means that files can be decompressed separately, or potentially even use different compression algorithms) - there's a patched version that supports zstd: https://github.com/mcmilk/7-Zip-zstd

                Note that you might also make a tar archive of compressed files, but that's less common.
                Last edited by gyscos; 01-08-2020, 03:29 PM.

                Comment


                • #9
                  Originally posted by willbprog177 View Post
                  Hope OpenMandriva's website starts working again soon. It's been 'off the air' for a few days at least. I don't do Facebook, so...
                  The hoster seems to have messed up routing pretty badly - if they don't fix it really soon now, we'll switch to another. Shouldn't take much longer...
                  Fakebook is pure evil, zstd is the only good thing that has ever come out of them.

                  Comment


                  • #10
                    Quick further comment about Zstd: for the data I'm interested in compressing I've just finished a load of comparisons... in a turn I really didn't expect, 7z (LZMA2) is actually more efficient than Zstd, regardless of compression level... in terms of time taken, they're about the same. But LZMA2 creates reliably smaller archives.

                    Comment

                    Working...
                    X