Why LLVM/Clang Was Ported To A Super Computer

Most often whenever writing about LLVM and its Clang C/C++ compiler front-end on Phoronix, within the forums is a flurry of comments from those in support of and against this modular compiler infrastructure. Some are against LLVM/Clang simply because its BSD-licensed and sponsored by Apple rather than the GPLv3-licensed GCC backed by the FSF. Others, meanwhile, see LLVM as presenting unique advantages and benefits. What reasons would a leading US national laboratory have for deploying LLVM/Clang to their leading super-computer? Here's an explanation from them.

The Argonne National Laboratory outside of Chicago presented at the LLVM Developers' Meeting earlier this month in California. The subject of this talk by one of the largest US national laboratories for scientific and engineering research was on "LLVM and Clang on the Most Powerful Supercomputer in the World."

As of the June "Top 500" super-computer numbers, the first and third spots were IBM Blue Gene/Q systems. The top spot was "Sequoia" at the LLNL while Argonne had "Mira" in its third place position. The top twenty positions meanwhile on the Green 500 super-computer list for leading performance-per-Watt were all Blue Gene/Q systems. Argonne National Laboratory had ported LLVM/Clang to their BG/Q super-computer.

The BG/Q architecture is 64-bit PowerPC-based with support for Quad-Vector Floating-Point (QPX) to extend regular PowerPC floating-point registers. Each BG/Q node is made of 16 user-accessible cores, one system core, and one spare core. The reasons expressed by Hal Finkel of the ANL laboratory for bringing LLVM/Clang to their super-computer came down to:

- Providing a high-performance and up-to-date C/C++ compiler over the life-time of the machine.

- Researchers at Argonne especially enjoy Clang's diagnostic abilities and static-analysis framework. Specifically noted were tagged-type diagnostics that will benefit the high-performance computing (HPC) community. "To my knowledge, no other compiler can produce these kinds of warnings, and these will be extremely valuable to our users."

- LLVM allows for tapping other languages that have an LLVM back-end like Intel's ISPC and different scripting languages.

- LLVM makes for compelling compiler research on the Blue Gene/Q architecture with its auto-vectorization support, parallelization with transactional memory and speculative execution, and communication-related optimizations and distributed systems.

Bringing LLVM and Clang to the Argonne National Laboratory's impressive super-computer involved modifying the PowerPC back-end, developing an itinerary for the A2-core based upon IBM's documentation, adapting the Hexagon hardware loops pass to work on PowerPC, and adding support for QPX intrinsics in Clang. Those were just the "easy" parts. The ANL researchers additionally developed a basic-block auto-vectorizer, modified the SelectionDAG builder, added support for v4i1 booleans to support QPX logical operations, cleaned-up/modernized the upstream PowerPC back-end, and fixed bugs within LLVM related to the labs' new target code.

With Argonne still being fond of LLVM/Clang, some of their future plans for improving the compiler infrastructure include support for generating all QPX instructions, more PowerPC back-end enhancements like better handling of condition registers and spilling, parallelization support (such as for LLVM/Clang's long-missing OpenMP support), higher level loop transformations with Polly, and MPI-specific optimizations. The national laboratory has a strong desire to make LLVM with Clang a "powerful force in HPC!"

There are PDF slides available of this ANL LLVM presentation for those interested in more details.