Announcement

Collapse
No announcement yet.

GCC 4.9 Compiler Optimization Benchmarks For Faster Binaries

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • GCC 4.9 Compiler Optimization Benchmarks For Faster Binaries

    Phoronix: GCC 4.9 Compiler Optimization Benchmarks For Faster Binaries

    For those curious about the impact of modern compiler tuning CFLAGS/CXXFLAGS when using the GCC 4.9 compiler with an Intel Core i7 "Haswell" processor, here are many benchmarks of many C/C++ code-bases when testing a variety of compiler optimization levels and other flags.

    http://www.phoronix.com/vr.php?view=20169

  • #2
    The LTO results seems wrong, like in the yesterday post.

    Michael, as discussed in the other thread, the LTO tests seems to be done without plugin support. Plugin is important, because it enables the compiler to make whole program assumptions transparently. Can you, please, ensure that you have recent binutils and plugin support (you can check by compiling with -fuse-linker-plugin and re-benchmark?

    To get decent compile times, you can also use -flto=n where n is number of parallel compilations.

    For those who are interested, some of Firefox benchmarks with and w/o LTO are at the end of this article http://hubicka.blogspot.ca/2014/04/d...-feedback.html

    Comment


    • #3
      Originally posted by hubicka View Post
      The LTO results seems wrong, like in the yesterday post.

      Michael, as discussed in the other thread, the LTO tests seems to be done without plugin support. Plugin is important, because it enables the compiler to make whole program assumptions transparently. Can you, please, ensure that you have recent binutils and plugin support (you can check by compiling with -fuse-linker-plugin and re-benchmark?
      As mentioned in the article, the builds failed with -fuse-linker-plugin, so Ubuntu 14.04 probably isn't shipping with new enough binutils?

      Originally posted by hubicka View Post
      To get decent compile times, you can also use -flto=n where n is number of parallel compilations.
      I also mentioned that in the article that -flto can be used with specifying thread count but for simplicity sake had just used it as -flto for the few compile benchmarks. The testing was mainly to get some fresh benchmarks out there at different optimization levels as opposed to focusing on LTO specifically.
      Michael Larabel
      http://www.michaellarabel.com/

      Comment


      • #4
        moar -march=native?

        Would have been nice to see -O1 & -O2 with -march=native to see the effect native has while being used with -O1-2 as well instead of only with -O3.

        Comment


        • #5
          Originally posted by HeavensRevenge View Post
          Would have been nice to see -O1 & -O2 with -march=native to see the effect native has while being used with -O1-2 as well instead of only with -O3.
          +1 (5 letters)

          Comment


          • #6
            Originally posted by Michael View Post
            As mentioned in the article, the builds failed with -fuse-linker-plugin, so Ubuntu 14.04 probably isn't shipping with new enough binutils?
            What failure did you get?

            One posibility is that plugin support was disabled in Ubuntu's binutils. This is easy to check by

            Code:
            $ ld --help | grep plugin
              --plugin PLUGIN             Load a plugin library
              --plugin-opt OPTION         Pass an option to the plugin
            It is not difficult to build custom binutils and configure with plugin enabled (i think it is --enable-plugin, but I am not 100% sure right now).

            Other possibility is that your benchmarks builds static libraries and you did not use gcc-ar, gcc-nm and gcc-ranlib wrappers. It is bit anoying to get Makefiles to use those, I simply use wrapper scripts on my path:
            Code:
            $ cat ar
            #! /bin/sh
            
            command=$1
            shift
            
            /aux/hubicka/binutils-install/bin/ar $command --plugin /aux/hubicka/trunk-install/libexec/gcc/x86_64-unknown-linux-gnu/4.9.0/liblto_plugin.so $*
            $ cat nm
            #! /bin/sh
            
            /aux/hubicka/binutils-install/bin/nm --plugin /aux/hubicka/trunk-install/libexec/gcc/x86_64-unknown-linux-gnu/4.9.0/liblto_plugin.so $*
            $ cat ranlib
            #! /bin/sh
            
            /aux/hubicka/binutils-install/bin/ranlib $command --plugin /aux/hubicka/trunk-install/libexec/gcc/x86_64-unknown-linux-gnu/4.9.0/liblto_plugin.so $*
            You may want to read -flto documentation in invoke.texi.

            Comment


            • #7
              I'm using LTO successfully with GCC 4.8 on both Ubuntu 13.10 and 14.04, but 13.10 is missing the soft links gcc-ar, gcc-ranlib, and gcc-nm but the binaries gcc-ar-4.8 etc. are installed.

              You can use this in a Makefile, instead of gcc-ar, etc:
              Code:
              LTOPLUGIN := $(shell gcc -print-search-dirs | grep install | awk '{print $$2 "liblto_plugin.so"}')
              ARFLAGS += --plugin $(LTOPLUGIN)
              RANLIBFLAGS += --plugin $(LTOPLUGIN)
              Build with -fno-fat-lto-objects and verify that lib*.a archives have correct LTO symbol indices with gcc-nm. Ordinary nm should only print:
              Code:
              0000000000000001 C __gnu_lto_slim
              0000000000000001 C __gnu_lto_v1

              Comment


              • #8
                Typo on page 4:
                but when optimizing for size (-O3) the performance was even slower
                It should have said "-Os" not "-O3".

                Comment


                • #9
                  Originally posted by AJenbo View Post
                  Typo on page 4:

                  It should have said "-Os" not "-O3".
                  Fixed, thanks.
                  Michael Larabel
                  http://www.michaellarabel.com/

                  Comment


                  • #10
                    A much better test would be compiling ... Mozilla Firefox with given optimization levels and then measuring its performance in various JS benchmarks.

                    I'm not aware of any automation tools for LibreOffice to test its performance, but LibreOffice tests would be nice as well.

                    The problem with these artificial benchmarks is that the average Joe rarely if never runs them.

                    The average Joe meanwhile spends a lot of time browsing the web and creating spreadsheets.

                    Comment


                    • #11
                      Originally posted by birdie View Post
                      A much better test would be compiling ... Mozilla Firefox with given optimization levels and then measuring its performance in various JS benchmarks.

                      I'm not aware of any automation tools for LibreOffice to test its performance, but LibreOffice tests would be nice as well.

                      The problem with these artificial benchmarks is that the average Joe rarely if never runs them.

                      The average Joe meanwhile spends a lot of time browsing the web and creating spreadsheets.
                      If there were such automated application tests, I'd be happy to run them, but alas there isn't.
                      Michael Larabel
                      http://www.michaellarabel.com/

                      Comment


                      • #12
                        Originally posted by Michael View Post
                        If there were such automated application tests, I'd be happy to run them, but alas there isn't.
                        And even if there was, the amount of time it takes to compile Firefox is already ridiculous enough!

                        Comment


                        • #13
                          Michael, could you please include -mtune=generic to get a glimpse how fast distribution provided binaries are? That would be quite interesting!

                          Comment


                          • #14
                            Originally posted by Michael View Post
                            If there were such automated application tests, I'd be happy to run them, but alas there isn't.
                            Firefox has quite few ways to benchmark it. Dromaeo, telemetry, talos, sunspider... I plan to get some more data, so far I only measured dromaeo with and without LTO/FDO http://hubicka.blogspot.ca/2014/04/d...-feedback.html

                            I also built libreoffice with and without LTO. W/o LTO the directory with all libraries is 430MB (GCC 4.9), with clang's February mainline checkout LTO it is 350 and with GCC 4.9 it is 303MB. I will discuss with LO folks ways to benchmark it. So far I only got suggestion to run the unit tests. Will do that as time allows.

                            Comment


                            • #15
                              Originally posted by hubicka View Post
                              Firefox has quite few ways to benchmark it. Dromaeo, telemetry, talos, sunspider... I plan to get some more data, so far I only measured dromaeo with and without LTO/FDO http://hubicka.blogspot.ca/2014/04/d...-feedback.html
                              But with any of those ways is it automated from launching Firefox to closing it and gathering the results? I've heard of a few ways to automate actions within the browser but unfortunately not a way for it to be fully automated where it could be launched from a script and the result returned upon completion.

                              Originally posted by hubicka View Post
                              I also built libreoffice with and without LTO. W/o LTO the directory with all libraries is 430MB (GCC 4.9), with clang's February mainline checkout LTO it is 350 and with GCC 4.9 it is 303MB. I will discuss with LO folks ways to benchmark it. So far I only got suggestion to run the unit tests. Will do that as time allows.
                              So far haven't been aware of any real ways to fully automate LO. Back in the OpenOffice days there was reportedly some automated benchmarks via http://www.oooninja.com/2009/03/mult...chmark-30.html and that author I had contacted said he would release the scripts eventually or sell them.
                              Michael Larabel
                              http://www.michaellarabel.com/

                              Comment

                              Working...
                              X