Announcement

Collapse
No announcement yet.

CompilerDeathMatch 64bit Final results

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • CompilerDeathMatch 64bit Final results

    I hereby present the final results from my compiler death match comparisons. Based on the experiences obtained during the generation of this dataset, I have decided to present it sorted and merged per optimization setting. If people are interested, I can also add merges of the data sorted per compiler (comparing different settings, I already have such data for PCC and Clang).

    All the comparisons are now finished, with the exception of "Custom" that upon request still can be changed further.

    DeathMatch round: Default
    DeathMatch round: O2
    DeathMatch round: O3
    DeathMatch round: Os
    DeathMatch round: Custom

  • #2
    There are missing compilers in some graphs. Worst example in the timed compilation of "custom".

    Comment


    • #3
      Originally posted by RealNC View Post
      There are missing compilers in some graphs. Worst example in the timed compilation of "custom".
      I am not entirely sure about what you mean. That some compilers are missing from some tests are often due to that they failed that test (for example the compilation tests, where it was common that I got a compilation time but with a message like "exited with a non-zero status" or something like that. Those results were automatically omitted by PTS).

      For Custom, I have only had requests for specific settings of GCC, ICC, Open64 and the flags I got as a tip from the PCC developer. PCC and TCC can not be included in the -O tests since they do not have those options (yet).

      Comment


      • #4
        What I mean is, for example the "Timed Apache Compilation" only lists GCC and ICC. But later on, in the "Apache Benchmark", it also lists Open64 and PCC.

        So from that one wonders how it's possible to do the "Apache Benchmark" with Open64 and PCC if those compilers weren't able to actually build Apache? (Since they're missing from the "Timed Apache Compilation" bench.)

        Comment


        • #5
          Originally posted by RealNC View Post
          What I mean is, for example the "Timed Apache Compilation" only lists GCC and ICC. But later on, in the "Apache Benchmark", it also lists Open64 and PCC.

          So from that one wonders how it's possible to do the "Apache Benchmark" with Open64 and PCC if those compilers weren't able to actually build Apache? (Since they're missing from the "Timed Apache Compilation" bench.)
          That is indeed a strange thing with the PTS tests. As I said in the previous reply, it often seemed like the compilations actually happened during compilation tests, but that they failed some sort of quality test and were omitted from the results. Perhaps the resulting binaries were still possible to use in the functional tests... The author of the "compiler" suite could probably explain this way better than I can.

          Comment


          • #6
            Ohhh, i like custom!
            Then gcc suddenly wins it all again. Look at the apache static page benchmark. With custom it gets a whopping 4117 requests per second! In your benchmarks i've never seen a higher number then that. I do wonder why open64 dropped... it seems to decrease performance with those flags.

            in fact, gcc seems to win (nearly) all with those flags which leads me to thing that the CO: CFLAGS has something odd in it since open64 can perform better at just -o2 ..

            Comment


            • #7
              Actually only TCC seems to get higher... I wonder what TCC is doing

              Comment


              • #8
                I found this by accident, but could you try adding this to the CG CFLAGS: -mfpmath=sse or even -mfpmath=both ... http://gcc.gnu.org/onlinedocs/gcc-4....4-Options.html

                Comment


                • #9
                  Originally posted by markg85 View Post
                  I found this by accident, but could you try adding this to the CG CFLAGS: -mfpmath=sse or even -mfpmath=both ... http://gcc.gnu.org/onlinedocs/gcc-4....4-Options.html
                  -mfpmath=sse should be the default setting on x86_64 according to this text...

                  Comment


                  • #10
                    Thanks again for the benchmarks. The graphs are prettier now.

                    wow! I agree, GCC with custom flags rocks!!

                    GCC apache (request per second)

                    custom: 4117
                    O2: 2478
                    O3: 2385

                    near 90% performance improvement ??

                    GCC 7-ZIP (MIPS)

                    custom: 2105
                    O2: 1731
                    O3: 1570

                    In general, It seems that GCC performs very well, I have made a few tests by my own of GCC vs ICC and I have found similar results, the only cases where ICC seems to really outperform GCC is on multi-threaded applications like mplayer-mt or c-ray-mt.

                    Comment

                    Working...
                    X