Announcement

Collapse
No announcement yet.

LLVM/Clang 3.2 Compiler Competing With GCC

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • LLVM/Clang 3.2 Compiler Competing With GCC

    Phoronix: LLVM/Clang 3.2 Compiler Competing With GCC

    With last week's release of LLVM 3.2, here are new benchmarks of LLVM 3.2 with the Clang C/C++ compiler front-end. The LLVM/Clang 3.2 performance using last week's source code releases were compared to the earlier LLVM/Clang 3.1 release and then for competition was the GCC 4.7.2 stable release and the latest GCC 4.8.0 development snapshot.

    http://www.phoronix.com/vr.php?view=18321

  • #2
    GCC Wins hands down

    GCC wins hands down. GO and cry Apple fanboys !!!

    Comment


    • #3
      Does anyone know how the two compare on ARM processors? This seems to be an area where the two compete head-to-head.

      Comment


      • #4
        Originally posted by wargames View Post
        GCC wins hands down. GO and cry Apple fanboys !!!
        A) Just supporting LLVM doesn't make someone an "apple fanboy" so go and take your apparent GCC-fanboyism somewhere else.

        B) "Hands down" is a BIG stretch. LLVM was extremely competitive in just about every test that doesn't require OpenMP, and once OpenMP IS sorted out I bet those benchmarks will show LLVM is about the same as GCC in those tests as well.

        C) The benefits in LLVM aren't in speed. 1) Modularity-- something GCC cant accomplish without scrapping a large chunk of the codebase SPECIFICALLY because GNU WANTED a monolithic codebase so that companies couldn't pull in select portions of gcc and just use those. 2) Cleaner code base. 3) Competition finally amongst compilers finally. GCC has gone uncontested in the open source world for a very long time, its good to see a project finally giving it a run for its money and ensuring that they don't start to rest on their laurels

        Comment


        • #5
          Originally posted by Ericg View Post
          A) Just supporting LLVM doesn't make someone an "apple fanboy" so go and take your apparent GCC-fanboyism somewhere else.

          B) "Hands down" is a BIG stretch. LLVM was extremely competitive in just about every test that doesn't require OpenMP, and once OpenMP IS sorted out I bet those benchmarks will show LLVM is about the same as GCC in those tests as well.

          C) The benefits in LLVM aren't in speed. 1) Modularity-- something GCC cant accomplish without scrapping a large chunk of the codebase SPECIFICALLY because GNU WANTED a monolithic codebase so that companies couldn't pull in select portions of gcc and just use those. 2) Cleaner code base. 3) Competition finally amongst compilers finally. GCC has gone uncontested in the open source world for a very long time, its good to see a project finally giving it a run for its money and ensuring that they don't start to rest on their laurels
          to 1) and this was and still is a viable reason
          to 2) can't judge... don't know it and do not care neither.
          to 3) well, as the test results show they did quite well over a lot of years even without competition. actually i think this shows quite well one major plus of oos, they do not really need competition to be pushed forward. imagine how bad a closed source compiler would have evolved without any competion? the few devs wouldn't care about issues...

          edit: before somebody gets me wrong: i do wellcome the new competition.

          Comment


          • #6
            Originally posted by a user View Post
            to 1) and this was and still is a viable reason
            to 2) can't judge... don't know it and do not care neither.
            to 3) well, as the test results show they did quite well over a lot of years even without competition. actually i think this shows quite well one major plus of oos, they do not really need competition to be pushed forward. imagine how bad a closed source compiler would have evolved without any competion? the few devs wouldn't care about issues...

            edit: before somebody gets me wrong: i do wellcome the new competition.
            No even Open Source Software absolutely needs competition, particularly when it's a monolithic product as opposed to a more modular one*. The only reason that GCC is somewhat faster at this point in time is that it's been being developed since 1987, LLVM on the other hand was started in 2003, which means GCC has 13 years of development time in advance of LLVM, that's more than twice the lifespan of LLVM.

            A perfect example of the competition bringing forth better products in OSS here with GCC being that GCC now has decent error reporting capability because of the LLVM people having done it. As well as a relatively large refactoring of the GCC codebase into C++ and otherwise.

            What we're seeing here though is the natural result in the difference in design between Modular and Monolithic architectures, a modular project make take a bit longer for initial design and setting up but once it's there it can accelerate far faster and far longer (almost if not indefinitely) whereas a monolithic application is much quicker to design and set up but without refactoring to a more modular design the development will slow eventually to a halt no matter how many people you throw at it, just because the codebase is so unwieldy, thus allowing a properly modularly designed application to accelerate past it.

            * Now theoretically a Completely Modular codebase could exist in a vaccuum and do quite well, but it still benefits if nothing else from pulling ideas from competing products

            Comment


            • #7
              Originally posted by a user View Post
              to 1) and this was and still is a viable reason
              to 2) can't judge... don't know it and do not care neither.
              to 3) well, as the test results show they did quite well over a lot of years even without competition. actually i think this shows quite well one major plus of oos, they do not really need competition to be pushed forward. imagine how bad a closed source compiler would have evolved without any competion? the few devs wouldn't care about issues...

              edit: before somebody gets me wrong: i do wellcome the new competition.
              GCC had major competition, what are you talking about? It's still the unpopular choice in a number of important fields where other compilers are dominant. Even in fields where it is dominant, the versions used are often forks of ancient crufty versions (in some cases 3.x, not even 4.x) with tons of vendor-specific changes (and the GPL is barely useful here since the FSF requires copyright assignment to roll code back into GCC proper, which said vendors have zero interest in granting, and hobbyist developers have little interest in patching, fixing, forking, or maintaining ancient vendor forks).

              Comment


              • #8
                I think it is safe to say that GCC has finally a competitor.

                Sure, they are released under different licences BUT we all agree that variety is man's wealth, do we?

                Comment


                • #9
                  I see quite a few useless test comparisons due to OpenMP not being on both compiler suites.

                  I look forward to useful test comparisons when OpenMP is a go into LLVM/Clang.

                  Comment


                  • #10
                    Originally posted by wargames View Post
                    GCC wins hands down. GO and cry Apple fanboys !!!
                    Ignorance is your bliss.

                    Comment


                    • #11
                      Originally posted by Luke_Wolf View Post
                      No even Open Source Software absolutely needs competition,
                      why "no"? did i claimed the opposite? no, i didn't! i just pointed out that in this case it worked well even without AND actually oos without competition has better chances to eveolve than closed source without competition.

                      this does not mean that competition in general is also for oos recommended.

                      particularly when it's a monolithic product as opposed to a more modular one*. The only reason that GCC is somewhat faster at this point in time is that it's been being developed since 1987, LLVM on the other hand was started in 2003, which means GCC has 13 years of development time in advance of LLVM, that's more than twice the lifespan of LLVM.
                      no, this is wrong! actually this has quite little to do with gcc's current state of performance. if your claim would be true than gcc wouldn't had performed so well in comparision with other compilers that existed over the years. just consider ms c++ compiler which actually performes quite bad, or compare it to the intel compiler, which used to perform better, though gcc still could hold up quite well.

                      the reason why llvm start to enhance so fast is that it could and did learned from the past of the others.

                      A perfect example of the competition bringing forth better products...
                      actually you don't need to post this here as i haven't seen somebody denieng that competition is good.

                      What we're seeing here though is the natural result in the difference in design between Modular and Monolithic architectures, a modular project make take a bit longer for initial design and setting up but once it's there it can accelerate far faster and far longer (almost if not indefinitely) whereas a monolithic application is much quicker to design and set up but without refactoring to a more modular design the development will slow eventually to a halt no matter how many people you throw at it, just because the codebase is so unwieldy, thus allowing a properly modularly designed application to accelerate past it.
                      you are simplifying things quite a lot. it is easy for interpret things like this while ignoring the fact that llvm can look back on the work of all the other compiler devs and learn from it.

                      * Now theoretically a Completely Modular codebase could exist in a vaccuum and do quite well, but it still benefits if nothing else from pulling ideas from competing products
                      i like how you are talking about a "completely modular codebase". there is no completely modular codebase, and even less a viable definition of such a thing. you can overmodularize things and make it worse. there is always the need of a design decision what and how much do you modularize. and afterwards you always see you could have done it better. LLVM has the advantage to look "afterwards".

                      this helps llvm to improve fast to the current level of gcc. but if it can top it significantly is yet to be seen.

                      Comment


                      • #12
                        Any particular reason why in a few benhcmarks, LLVM is FAR FAR ahead of GCC ? It almost looks like GCC is broken is these benchmarks.

                        LLVM is doing pretty good. It has narrowed the differences quite a lot. But its the last 10% difference thats the hardest to overcome.

                        Comment


                        • #13
                          Originally posted by mayankleoboy1 View Post
                          Any particular reason why in a few benhcmarks, LLVM is FAR FAR ahead of GCC ? It almost looks like GCC is broken is these benchmarks.

                          LLVM is doing pretty good. It has narrowed the differences quite a lot. But its the last 10% difference thats the hardest to overcome.
                          It most likely has to do with tunables - like at what point the compiler defaults to inlining methods versus using function calls, etc. Those are hard to get 100% right - there will always be some benchmarks that end up on the wrong side of the cutoff, but tweaking to speed them up may result in small losses across hundreds of other benchmarks and make it not worth it.

                          Comment


                          • #14
                            Originally posted by mayankleoboy1 View Post
                            Any particular reason why in a few benhcmarks, LLVM is FAR FAR ahead of GCC ? It almost looks like GCC is broken is these benchmarks.

                            LLVM is doing pretty good. It has narrowed the differences quite a lot. But its the last 10% difference thats the hardest to overcome.
                            I think you missed the part where in some charts, less is better.. While in other charts, more is better. LLVM is never far far ahead, it's far far behind in some tests. Supposedly due to lacking OpenMP support.

                            Comment


                            • #15
                              Originally posted by mayankleoboy1 View Post
                              Any particular reason why in a few benhcmarks, LLVM is FAR FAR ahead of GCC ? It almost looks like GCC is broken is these benchmarks.
                              You can dismiss any comparison where there isn't a -On flag set as we have no idea what optimization level is used. If you don't set a compiler optimization flag, GCC will deafult to -O0, which is no optimization.

                              Also, there's no way of ensuring that -O0,-O1,-O2 on GCC and Clang/LLVM are in any way equivalent optimization levels between compilers as there is no 'law' on what optimizations to enable at these levels. As such they are pretty worthless for compiler vs compiler benchmarks, atleast if you exclude -O3.

                              -O3 on the other hand is the highest optimization level which favors speed above everything else, so if you are only going to benchmark _one_ optimization level then this is it. There are also other special optimizations like link time optimizations (available in both GCC and Clang/LLVM), profile guided optimization (in GCC but not yet in Clang/LLVM) but these are beyond the scope of these tests in my opinion as Michael already have a hard time with the standard -On levels from the looks of it.

                              So looking at these tests, only the following declare setting the -O3 flag (best performant code, which is what we are benchmarking here):

                              Timed HMMER Search
                              Timed MAFFT Alignment
                              Blake2
                              Himeno Benchmark
                              C-Ray
                              Smallpt (OpenMP so rather pointless to compare)
                              GraphicsMagic (OpenMP so rather pointless to compare, how about configuring with --disable-openmp ?)
                              Tachyon (-m32 ? doesn't this work with 64-bit? )

                              Then we have Lame MP3 which defaults to -O3, so that should be ok unless Michael has changed the settings, however afaik this relies on assembly optimizations aswell.

                              JohnTheRipper defaults to -O2 (ignores CFLAGS environment variable so you need to manually edit makefile) and uses lots of assembly optimizations which makes it a poor candidate for benchmarking. However afaik it only uses OpenMP if you set OMPFLAGS to -fopenmp (not default), so why Michael is enabling it for this test is beyond me as it only makes sense to have if both compilers being benchmarked supports it.

                              VP8 libvpx and x264 would be great candidates for testing, both default to -O3 and offer a great challenge for optimizing compilers, except of course that Michael compiles them with assembly optimizations _enabled_ which makes them pointless as the compilers are left with pretty much no performance critical code to optimize, hence the pretty much identical results in these tests all these years.

                              The SciMark tests have no declared settings, and while Michael states that he sets CFLAGS/CXXFLAGS to -O3 -march=native, not all packages use/honor these environment variables, as seems to be the case here (as is the case with JohnTheRipper). Until we know the settings actually used in these tests they are worthless.

                              With Phoronix up for sale and with Michael indicating that he wants to continue focusing on the benchmarking I really hope takes the time to actually learn/take the time to properly configure the test software he benchmarks so that the results have better meaning.

                              Originally posted by mayankleoboy1 View Post
                              LLVM is doing pretty good. It has narrowed the differences quite a lot. But its the last 10% difference thats the hardest to overcome.
                              Yes there's obviously less low hanging fruit nowadays, obviously the lack of OpenMP is a big bottleneck and I don't understand why Michael persists in doing OpenMP based comparisons as of now at all as it's pointless until LLVM supports it. Just like it would be pointless to compare GCC + PGO against Clang/LLVM until it's supported in the latter.

                              Comment

                              Working...
                              X