No announcement yet.

The Bizarre Case Of Zstd's Very Slow Performance On Arch Linux

  • Filter
  • Time
  • Show
Clear All
new posts

  • #71
    Originally posted by mos87 View Post
    GNU autotools >>> CMAKE confirmed
    Autohell is not involved here. The Makefile is hand-written.


    • #72
      Originally posted by archkde View Post

      Autohell is not involved here. The Makefile is hand-written.
      On, I like these. Simple, easy to hack on, fast builds.


      • #73
        Originally posted by Beherit View Post
        Why is no one asking why the c99 flag was added, which is an older standard than both c11 and c18, when it’s obviously not required to compile zstd?

        Is this some kind of cmake or arch linux standard procedure done to all packages? Just trying to understand the reason behind it.
        Several years ago compilers defaulted to c89, so setting it to c99 was an improvement. Of course as compiler finally updated their C versions it became a regression instead.


        • #74
          Just want to note that Linux From Scratch downloads and compiles zstd from github and uses the default make file which only enables the CFLAGS="-std=c99 -Werror -O0" when you use the c99build target.

          I'm of two minds here, yes, the benchmark identified a miscalculation in multithreaded time calc, which is good, but also that it makes it sound like Arch has horrible performance for zstd compression when it really doesn't. I think after multiple times of getting these results it would have been good to do a basic timed test to see if the results are accurate.
          I don't use Arch, but I do reference their PKGBUILD files a lot when building additional packages or troubleshooting something when building my LFS systems.

          Michael - Thanks for what you do here with Phoronix! I appreciate all your individual hard work on providing the latest Linux news and for your work on phoronix-test-suite.

          Oops, guess I glanced over it too quick.. I just looked for the c99 and it looked like it was adding it by default, but I see now that you have to specify the c99build and then it adds those options. I wasn't able to actually check a build, but just looked at the Makefile at work. I retract my earlier statement.
          Last edited by skeetre; 20 June 2022, 04:59 PM. Reason: Didn't pay close enough attention to the makefile...


          • #75
            Just want to note that Linux From Scratch downloads and compiles zstd from github and uses the default make file which enables the CFLAGS="-std=c99 -Werror -O0"
            Clever - if you compile with -O0 you can as well screw up the benchmarks with -std=c99, because the performance will suck anyway :-p


            • #76
              > uses the default make file which enables the CFLAGS="-std=c99 -Werror -O0"

              Not sure where you got this information, but no, default Makefile doesn't enable any of these flags. It's essentially limited to -O3. No extra warnings, no extra errors, these are for debug builds.


              • #77
                Originally posted by jacob View Post

                I know Arch is not a source distro but you actually illustrate my point. You implicitly have to trust your distro no matter which one you use. If the distro provides binaries, you can trust that those binaries are good. Of course sometimes the packagers can get it wrong, like here, but it's the exception rather than the rule (as shown in this instance by the fact that all the other ones have good zstd builds). That's obviously no accident, because especially major distros like Ubuntu perform extensive testing *of the binaries they ship*. On a source distro (or with BSD ports etc) there is basically no way to keep that under control.
                Why wouldn't reproducibility allow for this? Gentoo, Arch, Debian... They can be made create the same exact binary from the sane exact code. Debian has mostly done this and I thought Arch had already too.


                • #78
                  Originally posted by jacob View Post

                  You are missing the point. Distros compile the source code and then test the binary. Except for rare cases when a problem slips through (like here) you know that the binary has been tested and vetted. On source oriented distros where everyone compiles everything differently, there is no QA of the binaries.
                  There could be with reproducibility. People tend to think it isn't important or useful but it is worth the effort to do what Debian and Arch to some extent, has done to make sure packages are reproducible. Gentoo could do it too.


                  • #79
                    Originally posted by mercster View Post
                    Imagine that, a "cool kid" rolling release distro has trouble keeping things together because they don't do enough QA, because building everything from source as soon as it comes out -funroll-loops -O999999 adds racing stripes to your Linux! Run Fedora or Ubuntu, kids. Trust me.
                    This is not what Archlinux is... and "building from source" is what every distro does.