HHVM Continues Beating PHP At Its Own Game

Given this week's release of HHVM 3.6 I decided to run some new performance comparison tests of PHP vs. HHVM.

As with my other recent PHP vs. HHVM performance comparisons, it was some basic command-line testing of both PHP implementations against our open-source Phoronix Test Suite benchmarking software. Under each implementation I ran phoronix-test-suite debug-self-test that's a basic test of some of the very common and hot code-paths of the Phoronix Test Suite. There's other PHP benchmarks available via the Phoronix Test Suite but for this very quick, one-page comparison I just ran this test -- and as a validation of our Git code ahead of the Phoronix Test Suite 5.6 release in the days ahead.

The hardware/software configuration for this quick testing was:

Phoronix Test Suite v5.6.0m5
System Information

Hardware:
Processor: Intel Core i7-5960X @ 3.50GHz (16 Cores), Motherboard: Gigabyte X99-UD4-CF, Chipset: Intel Xeon E5 v3/Core, Memory: 16384MB, Disk: 120GB Samsung SSD 850 + 120GB OCZ VECTOR150, Graphics: LLVMpipe, Audio: Intel C610/X99, Monitor: SyncMaster, Network: Intel Connection

Software:
OS: Ubuntu 15.04, Kernel: 3.18.0-12-generic (x86_64), Desktop: Unity 7.3.1, Display Server: X Server 1.16.2.901 (1.16.3 RC 1), Display Driver: radeon 7.4.0, OpenGL: 3.0 Mesa 10.5.0 Gallium 0.4, Compiler: GCC 4.9.2, File-System: ext4, Screen Resolution: 2560x1600

First I ran the test using the PHP binary currently shipped by Ubuntu 15.04 (Vivid), which was version 5.6.4-4ubuntu2:

PHP: 5.6.4-4ubuntu2
DETAILED_SYSTEM_INFO: 0.096 seconds
LIST_AVAILABLE_TESTS: 0.546 seconds
LIST_AVAILABLE_SUITES: 24.900 seconds
INFO: 0.017 seconds
CLONE_OPENBENCHMARKING_RESULT: 45.926 seconds
RESULT_FILE_TO_TEXT: 4.653 seconds
DIAGNOSTICS: 0.014 seconds
DUMP_POSSIBLE_OPTIONS: 0.002 seconds

ELAPSED TIME: 228.459 seconds
PEAK MEMORY USAGE: 35.75 MB
PEAK MEMORY USAGE (emalloc): 34.213 MB

228 seconds was the overall time for this test.

Next was the latest stable PHP.net release (PHP 5.6.6) configured with just --enable-zip and --enable-xml for handling our PHP CLI application:

PHP: 5.6.6
DETAILED_SYSTEM_INFO: 0.097 seconds
LIST_AVAILABLE_TESTS: 0.413 seconds
LIST_AVAILABLE_SUITES: 18.789 seconds
INFO: 0.015 seconds
CLONE_OPENBENCHMARKING_RESULT: 46.100 seconds
RESULT_FILE_TO_TEXT: 4.562 seconds
DIAGNOSTICS: 0.011 seconds
DUMP_POSSIBLE_OPTIONS: 0.001 seconds

ELAPSED TIME: 209.958 seconds
PEAK MEMORY USAGE: 35.75 MB
PEAK MEMORY USAGE (emalloc): 34.284 MB

With PHP 5.6.6 upstream it was down to 209 seconds overall.

I attempted to test PHP 7 Git this morning, but when testing out its fresh code, a segmentation fault was being produced straight away... So if you're interested in PHP 7 numbers until the segv issue is addressed, from recently there was this article of mine: Next-Generation PHP 7.0 Is Running Well But Will It Catch Up To HHVM?

Lastly was HHVM, which I used the pre-built Ubuntu 15.04 packages that were already bumped to 5.6.99:

PHP: 5.6.99-hhvm
DETAILED_SYSTEM_INFO: 0.299 seconds
LIST_AVAILABLE_TESTS: 0.215 seconds
LIST_AVAILABLE_SUITES: 10.002 seconds
INFO: 0.046 seconds
CLONE_OPENBENCHMARKING_RESULT: 16.604 seconds
RESULT_FILE_TO_TEXT: 0.022 seconds
DIAGNOSTICS: 0.037 seconds
DUMP_POSSIBLE_OPTIONS: 0.004 seconds

ELAPSED TIME: 81.688 seconds
PEAK MEMORY USAGE: 137.421 MB
PEAK MEMORY USAGE (emalloc): 14 MB

The reported peak memory usage was higher, but overall the test only took 81 seconds with HHVM in comparison to PHP 5.6 taking 209+ seconds! It's quite impressive and will be interesting to see how much they can gain this year out of integrating LLVM into HHVM. PHP 7 also remains quite interesting and I'll be back with more tests when its release nears.