Announcement

Collapse
No announcement yet.

CompilerDeathMatch 64bit Final results

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • CompilerDeathMatch 64bit Final results

    I hereby present the final results from my compiler death match comparisons. Based on the experiences obtained during the generation of this dataset, I have decided to present it sorted and merged per optimization setting. If people are interested, I can also add merges of the data sorted per compiler (comparing different settings, I already have such data for PCC and Clang).

    All the comparisons are now finished, with the exception of "Custom" that upon request still can be changed further.

    DeathMatch round: Default
    DeathMatch round: O2
    DeathMatch round: O3
    DeathMatch round: Os
    DeathMatch round: Custom

  • #2
    There are missing compilers in some graphs. Worst example in the timed compilation of "custom".

    Comment


    • #3
      Originally posted by RealNC View Post
      There are missing compilers in some graphs. Worst example in the timed compilation of "custom".
      I am not entirely sure about what you mean. That some compilers are missing from some tests are often due to that they failed that test (for example the compilation tests, where it was common that I got a compilation time but with a message like "exited with a non-zero status" or something like that. Those results were automatically omitted by PTS).

      For Custom, I have only had requests for specific settings of GCC, ICC, Open64 and the flags I got as a tip from the PCC developer. PCC and TCC can not be included in the -O tests since they do not have those options (yet).

      Comment


      • #4
        What I mean is, for example the "Timed Apache Compilation" only lists GCC and ICC. But later on, in the "Apache Benchmark", it also lists Open64 and PCC.

        So from that one wonders how it's possible to do the "Apache Benchmark" with Open64 and PCC if those compilers weren't able to actually build Apache? (Since they're missing from the "Timed Apache Compilation" bench.)

        Comment


        • #5
          Originally posted by RealNC View Post
          What I mean is, for example the "Timed Apache Compilation" only lists GCC and ICC. But later on, in the "Apache Benchmark", it also lists Open64 and PCC.

          So from that one wonders how it's possible to do the "Apache Benchmark" with Open64 and PCC if those compilers weren't able to actually build Apache? (Since they're missing from the "Timed Apache Compilation" bench.)
          That is indeed a strange thing with the PTS tests. As I said in the previous reply, it often seemed like the compilations actually happened during compilation tests, but that they failed some sort of quality test and were omitted from the results. Perhaps the resulting binaries were still possible to use in the functional tests... The author of the "compiler" suite could probably explain this way better than I can.

          Comment


          • #6
            Ohhh, i like custom!
            Then gcc suddenly wins it all again. Look at the apache static page benchmark. With custom it gets a whopping 4117 requests per second! In your benchmarks i've never seen a higher number then that. I do wonder why open64 dropped... it seems to decrease performance with those flags.

            in fact, gcc seems to win (nearly) all with those flags which leads me to thing that the CO: CFLAGS has something odd in it since open64 can perform better at just -o2 ..

            Comment


            • #7
              Actually only TCC seems to get higher... I wonder what TCC is doing

              Comment


              • #8
                I found this by accident, but could you try adding this to the CG CFLAGS: -mfpmath=sse or even -mfpmath=both ... http://gcc.gnu.org/onlinedocs/gcc-4....4-Options.html

                Comment


                • #9
                  Originally posted by markg85 View Post
                  I found this by accident, but could you try adding this to the CG CFLAGS: -mfpmath=sse or even -mfpmath=both ... http://gcc.gnu.org/onlinedocs/gcc-4....4-Options.html
                  -mfpmath=sse should be the default setting on x86_64 according to this text...

                  Comment


                  • #10
                    Thanks again for the benchmarks. The graphs are prettier now.

                    wow! I agree, GCC with custom flags rocks!!

                    GCC apache (request per second)

                    custom: 4117
                    O2: 2478
                    O3: 2385

                    near 90% performance improvement ??

                    GCC 7-ZIP (MIPS)

                    custom: 2105
                    O2: 1731
                    O3: 1570

                    In general, It seems that GCC performs very well, I have made a few tests by my own of GCC vs ICC and I have found similar results, the only cases where ICC seems to really outperform GCC is on multi-threaded applications like mplayer-mt or c-ray-mt.

                    Comment


                    • #11
                      Originally posted by Jimbo View Post
                      Thanks again for the benchmarks. The graphs are prettier now.

                      wow! I agree, GCC with custom flags rocks!!

                      GCC apache (request per second)

                      custom: 4117
                      O2: 2478
                      O3: 2385

                      near 90% performance improvement ??

                      GCC 7-ZIP (MIPS)

                      custom: 2105
                      O2: 1731
                      O3: 1570

                      In general, It seems that GCC performs very well, I have made a few tests by my own of GCC vs ICC and I have found similar results, the only cases where ICC seems to really outperform GCC is on multi-threaded applications like mplayer-mt or c-ray-mt.

                      Indeed. On the other hand, I think an interesting take-home-message from these benchmarks has been that while using normal flags, there are serious alternatives with comparable performance (most noteably: Open64 and Clang).

                      Comment


                      • #12
                        Originally posted by staalmannen View Post
                        Indeed. On the other hand, I think an interesting take-home-message from these benchmarks has been that while using normal flags, there are serious alternatives with comparable performance (most noteably: Open64 and Clang).
                        and that is with ext4! not the best filesystem for apache if you look at previous phoronix file system benchmarks.

                        But seriously, change the flags send to open64 since it can perform a lot better then you see now in the custom benchmark. Else just put it on -o2 since that was giving great results anyway.

                        Comment


                        • #13
                          Originally posted by markg85 View Post
                          and that is with ext4! not the best filesystem for apache if you look at previous phoronix file system benchmarks.

                          But seriously, change the flags send to open64 since it can perform a lot better then you see now in the custom benchmark. Else just put it on -o2 since that was giving great results anyway.
                          If someone comes up with better custom flags, I am willing to try them. The same goes for Clang.
                          I just took the flags suggested in the previous thread.

                          Comment


                          • #14
                            The results of open64 with custom flags are weird, it performs much better with O2.

                            Yeah! The normal O2 on open64 is indeed very impressive as you pointed, by a noticeable margin. Some distros devs should consider begin the use of open64, at least on CPU intensive applications.

                            Comment


                            • #15
                              Originally posted by Jimbo View Post
                              The results of open64 with custom flags are weird, it performs much better with O2.

                              Yeah! The normal O2 on open64 is indeed very impressive as you pointed, by a noticeable margin. Some distros devs should consider begin the use of open64, at least on CPU intensive applications.
                              If you got some other suggestions for optimizations I am all ears. If anyone got suggestions for Clang, that would be great too.

                              Comment

                              Working...
                              X