Benchmarking LLVM & Clang Against GCC 4.5

With the recent release of GCC 4.5 and the forthcoming release of LLVM 2.7 that is expected in the coming days, we have decided to see how the performance of GCC compares to that of LLVM. For this testing we have results from GCC 4.3/4.4/4.5 compared to LLVM with its GCC front-end (LLVM-GCC) and against Clang, which is the native C/C++ compiler front-end for the Low-Level Virtual Machine.

Beyond being used in unique ways such as by the Mono project and Gallium3D to optimize GPU shaders and provide a viable software rasterizer for Linux, the Low-Level Virtual Machine (LLVM) compiler infrastructure continues to gain interest among developers as a viable alternative to the GNU Compiler Collection. LLVM/Clang is gaining traction on FreeBSD, OpenBSD, and among Linux developers. GCC continues to be criticized for its large and old code-base that can be buggy, slow, and generating poor code.

Clang is the compiler front-end that uses LLVM behind it and provides support for C, C++, and Objective-C. The LLVM developers consider Clang as being production ready (and was tagged "Clang 1.0") for its C support with the release of LLVM 2.6 last year, but it is still under constant development. It was two months ago that LLVM's Clang was finally self-hosting in that it could properly build itself. LLVM 2.7 is due out on the 25th of April and it has much more work to Clang (it will be released as Clang 1.1) since the 2.6 release some months back. On the software side was an Ubuntu 10.04 LTS snapshot with the Linux 2.6.32 kernel.

Besides Clang there is LLVM-GCC, which is a modified version of GCC that provides a LLVM C front-end and guts out much of GCC. In September of last year we did publish GCC vs. LLVM-GCC benchmarks atop Mac OS X. With these older tests, it was found that LLVM-GCC was not quite as fast as GCC alone.

Today's testing takes our GCC 4.3.0/4.4.0/4.5.0 benchmarks from earlier this week and adds in LLVM-GCC and LLVM Clang 2.7 SVN results. Originally, we had built LLVM from source using a Subversion snapshot over the weekend, but there were some issues with it so we reverted to using the LLVM/Clang packages supplied in the Ubuntu Lucid repository. These packages include an LLVM 2.7 SVN snapshot from 2010-04-09, an LLVM 2.7 GCC 4.2 snapshot from 2010-03-17 and Clang from 2010-03-17. We were using 64-bit software in all of our testing.

Again the test system we used was a Lenovo ThinkPad T61 notebook with an Intel Core 2 Duo T9300 "Penryn" processor clocked at 2.50GHz, 4GB of DDR3 system memory, a 100GB Hitachi HTS72201 SATA HDD, and a NVIDIA Quadro NVS 140M graphics processor. We used effectively the same set of tests in our earlier GCC benchmarking, except there are a few tests that were removed either due to build issues under LLVM-GCC or LLVM-Clang or the test not being written in C/C++. The tests included measuring the compilation times of Apache, PHP, and ImageMagick. The tests we used to look at the generated binary performance were Apache, 7-Zip, LAME, Gcrypt, C-Ray, BYTE, GraphicsMagick, Himeno, John The Ripper, and HMMer. This testing was driven by the Phoronix Test Suite.