GCC vs. LLVM-GCC Benchmarks

Last Friday we published Mac OS X 10.6 benchmarks and then on Monday they were joined by Ubuntu 9.10 vs. Mac OS X 10.6 benchmarks. One of the requests that has come up since publishing those articles are to carry out a set of tests comparing the performance of LLVM and LLVM-GCC. With Apple's Snow Leopard release, some parts of the operating system were built using LLVM-GCC for optimized performance, although this compiler is not yet matured. In this article we have a set of 12 benchmarks comparing GCC to LLVM-GCC.

LLVM-GCC is a C front-end for the Low-Level Virtual Machine but its back-end is a modified version of GCC (4.2). LLVM has its own compiler front-end, clang, for C and C++, but it is not nearly as complete or mature as the GNU Compiler Collection. Clang has been improving in recent times, but some portions are still incomplete (largely with C++). LLVM-GCC is provided with Xcode 3.2 found in Mac OS X 10.6, but it can also be found in several distribution package repositories, including Ubuntu. Apple though is the primary sponsor of the Low-Level Virtual Machine. LLVM/Clang is actually replacing GCC in FreeBSD Base. While we are not testing Clang in this article, for a comparison of Clang against GCC, read LLVM's comparison. Additional information on this open-source compiler infrastructure is available from the project's web-site. LLVM is also used by Apple in their OpenCL implementation and is finding uses on Linux within Gallium3D.

For this benchmarking we used several test profiles from the Phoronix Test Suite that are written in C and are built from source during their installation procedure. Prior to running one of the tests, we set the default compiler to GCC 4.2.1 (Apple Build 5646) and then set the default compiler to LLVM-GCC (GCC 4.2.1, Apple Build 5646, LLVM Build 2118) prior to reinstalling the tests. With the resulting binaries from GCC and LLVM-GCC, we then looked at their performance. Compiler flags and other settings were maintained the same. These tests included LAME MP3 encoding, dcraw, OpenSSL, BYTE Unix Benchmark, John The Ripper, timed MAFFT alignment, Crafty, TSCP, Tachyon, and C-Ray.

The system we used to conduct these tests was a newer Mac Mini with an Intel Core 2 Duo processor clocked at 2.00GHz, a NVIDIA MCP79 motherboard with GeForce 9400M graphics, 1GB of DDR3 memory, and a 120GB Fujitsu MHZ2120BH G1 SATA hard drive. Mac OS X 10.6.0 was used with its 10.0.0 kernel, X Server 1.4.2, OpenGL 2.1 NVIDIA-1.6.0, and a Journaled HFS+ file-system.