Facebook's HipHop Can Now Build Grimstad

Announced to the public last year by Facebook was HipHop, an open-source project that transforms PHP code into highly-optimized C++ and then uses the GCC C++ compiler to produce a native system binary. Facebook's original numbers showed that by using this transformer/compiler on their servers the CPU usage went down by about 50% and they were able to supply around 70% more traffic on existing resources since the PHP code is no longer being dynamically interpreted. Here's a look at Facebook's HipHop during some of our first tests.

Facebook developed HipHop so they could continue writing the core of Facebook's code in PHP as a scripting language without reverting to writing C/C++ extensions to PHP, which would mean hooking into Zend APIs and much more of a burden to their engineers. Facebook had also rewritten parts of PHP's Zend engine for increased performance, but that didn't lead to the performance benefits they were after. In the end, they wrote HipHop to transform PHP source code into C++ and then to build it under GCC.

The idea itself isn't original as the largely defunct Roadsend PHP / Roadsend Raven project had a similar goal of producing highly-optimized native system binaries and various other features, but it was leveraging the Low-Level Virtual Machine rather than GCC.

When Facebook's HipHop work was publicly announced in February of 2010 it received some attention by the media and others for "turbo charging PHP", but since then there hasn't been too much public interest that we've seen. However, a side project of mine for the past number of weeks has been to get Facebook's HipHop successfully building and running the Phoronix Test Suite. In particular, the Phoronix Test Suite 3.2-Grimstad code. Today it's successful in doing so.

While the Phoronix Test Suite is unique in that it's a widely-used non-server software package that's written in PHP as opposed to C/C++, Python, or even Java, PHP has worked out very well with the Phoronix Test Suite for its purposes. The quest to get the Phoronix Test Suite working under HipHop though hasn't been for any performance reasons, since pts-core (the Phoronix Test Suite engine) is rather light and not demanding at all: it's the actual test profiles being executed as the benchmark that's what's more taxing on the system. The Phoronix Test Suite just manages the test installation, execution, and the results.

So why even bother with building the Phoronix Test Suite client under HipHop? It's been a low priority task for a few reasons: there's been some that don't like PHP (or can't have it installed) on their systems, there's some environments where PHP can be troublesome to get running (such as some embedded/mobile cases), there may be some performance benefits to running the Phoronix Test Suite server code and the bilde_renderer / pts_Graph components of OpenBenchmarking.org, and it's just a research experiment.

Right now though Facebook's HipHop is limited to Linux x86_64 support, so it doesn't do much good for bringing the Phoronix Test Suite client to any new platforms where PHP is not already supported. It would be nice to produce native system binaries for ARM, but there's no sign of that support coming to HipHop soon or until Facebook begins evaluating a roll-out of any ARM servers. So right now this initial HipHop upbringing is more about an experiment and seeing if there would be any performance gains to the OpenBenchmarking.org server infrastructure as its features and capabilities become enriched.

While HipHop seeks to implement the entire PHP language, it wasn't the most straightforward process to build all of the Phoronix Test Suite (pts-core, bilde_renderer, pts_Graph, nye_Xml, etc) under HipHop. At this time Facebook is targeting PHP 5.2 transformations via HipHop while PHP 5.3 is out on the horizon. That's good enough for the Phoronix Test Suite as it will run fine going back to early PHP 5.1.x builds, but there's some functions (e.g. the eval() call) and other items that HipHop doesn't yet implement.

At the same time, there's a few areas where HipHop's transformation of the PHP code to C++ is less than ideal. As a result, a few areas of the Phoronix Test Suite code were reworked and cleaned up in the process so that it can be more efficiently and directly translated to C++. This work is living in Git master for Phoronix Test Suite 3.2-Grimstad. The latest code on Phorogit as of today (8 May) should successfully build and work with the latest HipHop code. This work will also be found with Phoronix Test Suite 3.2 Milestone 3 to be released in the near future. Documentation on how to build the Phoronix Test Suite with Facebook's HipHop will also be included. To those who don't care about HipHop, the Phoronix Test Suite will continue to work in its current form, as always. This is just an optional way for those who may want to run the Phoronix Test Suite as a native system binary.

For those curious about HipHop, here's some statistics. In its normal form, the Phoronix Test Suite is distributed as a 416kb .tar.gz package made up of roughly 38,709 lines of PHP code. When compiled under the latest HipHop / GCC 4.4.5 on Ubuntu 10.10 x86_64, the generated Phoronix Test Suite program binary is 33.7MB in size. The HipHop transformation from PHP to C++ turns the 38,709 lines of code into 161,028 lines of C++ code that is 6.8MB in raw form with another 20,339 lines of header files.

More information about Facebook's HipHop open-source project can be learned from its GitHub page.