Announcement

Collapse
No announcement yet.

CompilerDeathMatch: surprising results

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • #31
    nice work, staalmanen. Once you got your cflags tweaked for each compiler based on the input the other postes have provided you that produces the best optimizations then why not then do a final benchmark run?

    Also the unoptimized benchmarks (doing the compiles without any optimizations of special flags other than the architecture) are useful as a baseline.

    Comment


    • #32
      WOW, i never knew there are so any compilers for linux!
      I actually only knew gcc, icc and clang!

      But what strikes me more is the fact that for every task GCC does there seems to be a compiler out there that can do it faster and that can make a faster binary! In your benchmarks gcc never wins anywhere!

      However in that impressive chart i see one thing that just shocks me. APACHE static webpage serving! It seems, from your chart, that no mather how high you tune gcc, TCC and Open64 will nearly double the performance!

      Something odd is going on in your C-Ray benchmark.. every other benchmark has wildly varying scores, but C-Ray is nearly the same on every compiler.. That can't be right. (or is it..?)

      And if i look at it the right way it seems that using Open64 -o2 is beating ALL gcc benchmarks at -o2 and with some big differences as well! Now i wonder, why isn't everything compiled in Open64 since it obviously seems superior to GCC..

      Comment


      • #33
        Originally posted by markg85 View Post
        WOW, i never knew there are so any compilers for linux!
        I actually only knew gcc, icc and clang!

        But what strikes me more is the fact that for every task GCC does there seems to be a compiler out there that can do it faster and that can make a faster binary! In your benchmarks gcc never wins anywhere!

        However in that impressive chart i see one thing that just shocks me. APACHE static webpage serving! It seems, from your chart, that no mather how high you tune gcc, TCC and Open64 will nearly double the performance!

        Something odd is going on in your C-Ray benchmark.. every other benchmark has wildly varying scores, but C-Ray is nearly the same on every compiler.. That can't be right. (or is it..?)

        And if i look at it the right way it seems that using Open64 -o2 is beating ALL gcc benchmarks at -o2 and with some big differences as well! Now i wonder, why isn't everything compiled in Open64 since it obviously seems superior to GCC..

        As I said in my first post in this thread: "surprising results"

        Back then I ran the "out of the box" performance of the different compilers (similar to the recent Phoronix benchmark) and got completely surprised about the absolutely superior behavior of TCC being both FAST and producing very fast binaries.
        I try to run these things as unbiased as possible, but a part of me is somewhat pleased to see that there is serious competition to GCC out there since I think that diversity is a good thing.
        It would be very interesting to hear whether there are Gentoo (or other source-based distribution) users out there that are running complete Clang-compiled systems (via the LLL tool chain or similar) or perhaps even Open64-compiled systems.
        I have tried to google whether anyone has succeeded compiling linux with Open64 but did not find any evidence of that though...

        Comment


        • #34
          Originally posted by staalmannen View Post
          As I said in my first post in this thread: "surprising results"

          Back then I ran the "out of the box" performance of the different compilers (similar to the recent Phoronix benchmark) and got completely surprised about the absolutely superior behavior of TCC being both FAST and producing very fast binaries.
          I try to run these things as unbiased as possible, but a part of me is somewhat pleased to see that there is serious competition to GCC out there since I think that diversity is a good thing.
          It would be very interesting to hear whether there are Gentoo (or other source-based distribution) users out there that are running complete Clang-compiled systems (via the LLL tool chain or similar) or perhaps even Open64-compiled systems.
          I have tried to google whether anyone has succeeded compiling linux with Open64 but did not find any evidence of that though...
          Compiling the linux kernel "seems" to be possible with Open64 ... http://forums.amd.com/forum/messagev...hreadid=138368 (big LOL that i find that on the AMD forum the place i certainly didn't expect it)

          Comment


          • #35
            There's two odd things in this benchmarks..
            1- PCC don't complete 'timed apache compilation' but it appears in the 'static web page serving'.
            2- Clang seems to compile faster when optimization is turned on.

            Comment


            • #36
              Originally posted by tessio View Post
              There's two odd things in this benchmarks..
              1- PCC don't complete 'timed apache compilation' but it appears in the 'static web page serving'.
              2- Clang seems to compile faster when optimization is turned on.

              That is indeed strange. I do not really know how these tests are designed. I just started off with running the "compiler" suite and got data that later led to more data and so on...
              Perhaps the compilation time test is stricter on getting a "success" returned in order to approve the result, whereas the functional test of the resulting binary is more lenient when it comes to what the compilation returns?

              Comment


              • #37
                Cross post: Updated with -Os flags

                A new merge with the -Os results are hereby presented:

                http://global.phoronix-test-suite.co...-231-8340-1312

                Enjoy!

                Next up: Aggressive and compiler-specific settings (are there any tweaks that can be made to TCC?).

                Comment


                • #38
                  Originally posted by staalmannen View Post
                  A new merge with the -Os results are hereby presented:

                  http://global.phoronix-test-suite.co...-231-8340-1312

                  Enjoy!

                  Next up: Aggressive and compiler-specific settings (are there any tweaks that can be made to TCC?).
                  erm, wrong link....

                  Comment


                  • #39
                    nevermind, i found it. Here is the link: http://global.phoronix-test-suite.co...-231-8340-1312

                    Comment


                    • #40
                      SolarisStudio

                      I just downloaded SolarisStudio as well. Do people find it interesting that I also include this one in the set-up? This will delay the final closure of this experiment (and delay the start of 32-bit tests) further unfortunately, but would give it a wider scope.

                      Next up are compiler-specific flags none the less, and I will also try to rename the identifiers on all my experiments prior to the merge so that they get a nicer formatting (which definietly is needed now since there are so many data points).

                      Comment

                      Working...
                      X