LLVM Clang 3.9 Mostly Trails GCC In Compiler Performance

[phoronix]

Phoronix: LLVM Clang 3.9 Mostly Trails GCC In Compiler Performance

Following yesterday's GCC 5 vs. 6 vs. early 7 benchmarks, to no surprise LLVM's Clang compiler was brought up in the comments. I had already been running some fresh LLVM Clang benchmarks on this same Intel Xeon system and have those results to share now with Clang 3.8 and the newly-released Clang 3.9.

http://www.phoronix.com/vr.php?view=23521

[lowflyer]

I am really surprised that the compile times of Clang are not better. This is the area where Clang had the real lead over GCC.

[mlau]

I am really surprised that the compile times of Clang are not better. This is the area where Clang had the real lead over GCC.

me too, actually, although it shows that more optimization passes don't come for free.

[mlau]

I am also surprised by the relatively low scores of this Xeon processor. My haswell laptop's (@3.2GHz turbo) scimark2 scores are slightly more than double of what this 8-core Xeon @1.7GHz masters. I was actually expecting the Xeon with its larger caches and higher memory bandwith to do much much better...

[Del_]

I am also surprised by the relatively low scores of this Xeon processor. My haswell laptop's (@3.2GHz turbo) scimark2 scores are slightly more than double of what this 8-core Xeon @1.7GHz masters. I was actually expecting the Xeon with its larger caches and higher memory bandwith to do much much better...

Sciencemark has been accused of being a microbenchmark, i.e., being not very representative of real world loads. Your observation seems to support this. The benchmark may fit in your cache (nullifying the effect of larger cache or better memory bandwidth), it may have a simple instruction sequence which scales perfectly with clock speed of the CPU, and lastly, it may stress your system so little that it can run on the turbo frequency all the way through.

[discordian]

It would be interesting to run these tests on clang with the -fno-math-errno flag.
clang prohibits several optimization by default (which could result in visible differences), whereas gcc doesnt.

double a = sin(x); /* this could set errno */
double b = log(y); /* this could set errno */
double c = sin(x) + a; /* this could set errno */

gcc will only call sin(x) once and double the value, clang will not change the order since the functions could modify global state (errno) and reordering them could break programs that depend on this behavior.

Whetstone for example is notoriously affected by such flags.

[mlau]

Sciencemark has been accused of being a microbenchmark, i.e., being not very representative of real world loads. Your observation seems to support this. The benchmark may fit in your cache (nullifying the effect of larger cache or better memory bandwidth), it may have a simple instruction sequence which scales perfectly with clock speed of the CPU, and lastly, it may stress your system so little that it can run on the turbo frequency all the way through.

Oh I agree that it's a almost meaningless benchmark, but it at least shows where both compilers excel at code generation.
I ran the scimark2 suite with clang HEAD from a few hours ago, and it absolutely destroys gcc in the sparse matmult benchmark:

Sparse matmult Mflops: 12550.17 (N=1000, nz=5000) (clang git head)
Sparse matmult Mflops: 3118.18 (N=1000, nz=5000) (gcc-6.2)

There's no meaningful difference in the other scores.

[mlau]

It would be interesting to run these tests on clang with the -fno-math-errno flag.
clang prohibits several optimization by default (which could result in visible differences), whereas gcc doesnt.

double a = sin(x); /* this could set errno */
double b = log(y); /* this could set errno */
double c = sin(x) + a; /* this could set errno */

gcc will only call sin(x) once and double the value, clang will not change the order since the functions could modify global state (errno) and reordering them could break programs that depend on this behavior.

Whetstone for example is notoriously affected by such flags.

I did, and there's no difference in generated code at all, with both gcc and clang.

[s_j_newbury]

Oh I agree that it's a almost meaningless benchmark, but it at least shows where both compilers excel at code generation.
I ran the scimark2 suite with clang HEAD from a few hours ago, and it absolutely destroys gcc in the sparse matmult benchmark:

Sparse matmult Mflops: 12550.17 (N=1000, nz=5000) (clang git head)
Sparse matmult Mflops: 3118.18 (N=1000, nz=5000) (gcc-6.2)

There's no meaningful difference in the other scores.

Can you look at the generated code in each case and work out why?

[SilverMachine]

It's a shame you didn't include LLVM+CLANG 4.0.0.-SVN too.