Clang's Competition For GCC On Intel Haswell

[Sergio]

So, FUD goes on from Phoronix. It's at least second time when Phoronix makes unfair benchmarks in favor to clang.

So, you see, it is enabled by default.

Do you have any other objection?

[__UB]

Can someone independently repeat and confirm these measurements?

I have installed clang-3.4 (Fedora 20 version) from the llvm.org website

and gcc SVN "gcc version 4.9.0 20140205 (experimental) [trunk revision 207513] (GCC)".

Compilation with "-O3 -lm -march=native".

The results on IvyBridge Core i7 Extreme:

clang:
Composite Score: 1851.21
FFT Mflops: 1215.40 (N=1024)
SOR Mflops: 1547.32 (100 x 100)
MonteCarlo: Mflops: 523.78
Sparse matmult Mflops: 2109.55 (N=1000, nz=5000)
LU Mflops: 3859.99 (M=100, N=100)

gcc:
Composite Score: 1855.94
FFT Mflops: 1547.60 (N=1024)
SOR Mflops: 1546.66 (100 x 100)
MonteCarlo: Mflops: 550.19
Sparse matmult Mflops: 2040.82 (N=1000, nz=5000)
LU Mflops: 3594.40 (M=100, N=100)

However, it should be noted that the results were *EXTREMELY* unstable. In the warm-up run, clang scored only:

Composite Score: 1180.28
FFT Mflops: 433.27 (N=1024)
SOR Mflops: 1547.25 (100 x 100)
MonteCarlo: Mflops: 523.85
Sparse matmult Mflops: 2109.53 (N=1000, nz=5000)
LU Mflops: 1287.53 (M=100, N=100)

It looks that short loops in SciMark 2.0 tests are susceptible too much on the processor state (pre-cached data from previous runs). Even SciMark itself recognizes this deficiency and offers "-large" runtime switch. But then the benchmark will test the efficiency of caching and memory subsystem.

Just retire this thing and use real SPEC benchmarks.

[bulletxt]

I think it's safe to say that these benchmark are not worth anything. I do however appreciate Michael's work.

[Guest]

what a surprise!

No it's not, it's testing the defaults. It's a choice Clang chose to make for -O3 and afaik, -ftree-vectorize is on GCC for -O3.

Thanks for the explanation, but I bet most people think it's a fair comparison. It will be nice to put such info in the article.

[Guest]

So, you see, it is enabled by default.

Do you have any other objection?

Yes, I'd like you to stop posting stupid comments. So far everyone thought only distrubutions benchmarks were made with default settings. This is somehow understandable, but testing compiler defaults looks strange.

[Sergio]

Yes, I'd like you to stop posting stupid comments. So far everyone thought only distrubutions benchmarks were made with default settings. This is somehow understandable, but testing compiler defaults looks strange.

You complained because supposedly GCC did not use -ftree-vectorize by default, and then Michael proved that indeed it does; the confusion was due to a "bug in the help message dump" (did you even read Michael's response?). So I ask, is there any other objection about the tests? If so, could you elaborate?

I would have love to see the response if GCC would've kicked clang's ass; as always, when Linux/GNU software wins everybody take pride and talk about just how unbeatable they are. When they lose the benchmarks are useless.

Clang is already in pair with GCC. Furthermore, LLVM is the present and the future in the compiling/virtual machine industry and research. DEAL WITH IT.

[wizard69]

Thanks for the explanation, but I bet most people think it's a fair comparison. It will be nice to put such info in the article.

It is fair. Further if you read the entire article you would realize how the compilers are being tested. Really your whining here is pathetic, both compilers are performing really well. There are other reasons beyond performance to prefer CLang of over GCC though.

Note the word prefer, a smart developer these days might be using both tools to maximize error detection and problem areas. That is right, it is more important that a compiler helps the developer produce well tested and debugged code. Things like meaningful error reporting, or even a static checker can do wonders for the quality of code produced. Dwelling on performance numbers that often vary by fractions of a percent is a waste of time.

[wizard69]

Yes, I'd like you to stop posting stupid comments. So far everyone thought only distrubutions benchmarks were made with default settings. This is somehow understandable, but testing compiler defaults looks strange.

You have to be kidding!

[computerquip]

I would have love to see the response if GCC would've kicked clang's ass; as always, when Linux/GNU software wins everybody take pride and talk about just how unbeatable they are. When they lose the benchmarks are useless.

Clang is already in pair with GCC. Furthermore, LLVM is the present and the future in the compiling/virtual machine industry and research. DEAL WITH IT.

Nobody takes pride in the compiler they use unless they developed for it themselves. Rather, most developers go out of their way to support multiple compilers, including proprietary ones such as VC++ or Comeau. I don't mind stereotypes but you don't really have much of a basis for this one.

People on Phoronix hardly represent the majority of the Linux community or its developers. I personally hope LLVM/Clang ends up eventually replacing GCC because of its modular design and potential application (e.g. I've been wanting to experiment in emulation and JIT with LLVM), even though I develop for Linux almost in a biased manner.

[carewolf]

Thanks for the explanation, but I bet most people think it's a fair comparison. It will be nice to put such info in the article.

In this case the options are relatively similar, making it fair. It is a different talk when comparing to the Intel compiler that defaults to fast-math.