Linux 3.5 Through Linux 4.4 Kernel Benchmarks: A 19-Way Kernel Showdown Shows Some Regressions

What better way to spend a cold Friday morning than looking at some kernel benchmarks, so up for your viewing pleasure today are benchmarks of every kernel major release going from the Linux 3.5 kernel up through the latest Linux 4.4 stable kernel release. All the tests were done on the same system and there are actually some interesting performance changes to note with these Linux kernel tests going back to the summer of 2012.

The system I used this week for carrying out this Linux 3.5 to Linux 4.4 kernel comparison was the Xeon E5-2687W v3 Haswell processor (10 cores plus Hyper Threading), MSI X99S SLI PLUS motherboard, 16GB of DDR4 memory, a PNY CS121 120GB solid-state drive, and AMD FirePro V7900 (Cayman) graphics card. All of the hardware was maintained the same throughout testing and each kernel was tested with its defaults as obtained from the Ubuntu Mainline Kernel PPA archive; so you can fetch from there if you are curious about any of the Kconfig and other defaults.

A development snapshot of Ubuntu 16.04 x86_64 was the base operating system with Unity 7.4, X.Org Server 1.17.3, Mesa 11.0.8, GCC 5.3.1 compiler, and an EXT4 file-system.

I had intended to go back further than Linux 3.5, but when trying to boot Linux 3.4 or older, systemd wouldn't allow the system to boot. Thus Linux 3.5 was the furthest I could go back on this system.

Additionally, around Linux 3.11 is the oldest kernel I could go back for graphics tests. When trying to boot the kernel on Linux 3.11 or older with the Radeon DRM enabled, there seemed to be some clashes with the user-space driver stack and was only able to get a working desktop by passing "nomodeset" and using LLVMpipe.

The only other caveat to note in this testing is that the Linux 3.9 kernel couldn't boot on this Haswell Xeon system, so that kernel was skipped.

Thus we're left with a 19-way Linux 64-bit kernel comparison from this Intel Xeon Haswell system with SSD and FirePro V7900 graphics. Tests for this kernel comparison included SQLite, Dbench, Compile Bench, OpenArena, Tesseract, Xonotic, TTSIOD 3D Renderer, x264, Himeno, timed Linux kernel compilation, Parallel BZIP2 compression, FLAC audio encoding, LAME MP3 encoding, OpenSSL, PostgreSQL, and Redis.

All of these Linux kernel benchmarks were carried out in a fully-automated manner via the open-source Phoronix Test Suite benchmarking software. For those new to the site or haven't used the Phoronix Test Suite, our automated benchmarking software runs each test a minimum of three times to ensure accuracy (and keeps track of the standard deviation) and will dynamically increase the run-count if the results are fluctuating outside a defined threshold. Plus there are other features for ensuring statistical accuracy.

Some quick notes to point out before getting to the results is that with Linux 3.5 the kernel defaulted to the CFQ I/O scheduler while on all newer kernels it was with DEADLINE. For Linux 3.11 and older, ACPI CPUfreq was the default CPU scaling driver while on Linux 3.12 and newer this E5-2687W v3 system defaulted to Intel P-State with its performance governor. The EXT4 file-system mount options were the same across all of the tested kernels.