The Huge Disaster Within The Linux 2.6.35 Kernel

The performance setback with the computational biology HMMer test is huge. The Atom Z520 setup took 87.5% more time since the regression(s) was introduced while the Atom 330 had a lower but still significant 30% change.

The performance drop witnessed by the LU.A test within NASA's NAS Parallel Benchmarks is hugely devastating and the most significant. With the kernel code prior to last week the LU.A test consistently had around ~640 Mop/s but after the 22nd of May, this number has since become around ~215 Mop/s.

While you can find many more examples of this severe 2.6.35 regression on the Phoromatic Tracker for the Linux kernel, the last one we are sharing in this article is with the x264 video encoding test. Both of the systems here had dropped by very significant margins on the same dates as the others.

Since there is still some people that do not seem to trust the results of the Phoronix Test Suite / Phoromatic for whatever reasons, we manually confirmed this very serious Linux 2.6.35 kernel problem on an entirely different system. For those not familiar with our automated testing software, each test is run multiple times and if the standard deviation between the runs exceeds a predefined threshold, the run count will dynamically increase. Additionally, all included test profiles are designed to be deployed in a turnkey fashion and be fully reproducible (see The Five Stages of Benchmark Loss for some historical information).

For verifying this Linux 2.6.35 kernel problem we used a ZaReason Verix notebook that we are currently reviewing we did a clean installation of Ubuntu 10.04 LTS (x86_64) and then using the Ubuntu mainline daily kernel PPA had installed the 64-bit kernels from the 20th of May and then again from the 26th. For reference, there are no major changes in the kernel configuration between the two dates. We ran a few tests to just verify this is indeed a real, major regression and is not isolated to a couple Atom systems. The ZaReason Verix boasts an Intel Core i7 Q720 processor that's clocked at 1.60GHz with four physical cores plus Hyper Threading, has 6GB of system memory, sports an 80GB Intel SSD, and its display is powered by a NVIDIA GeForce GTS 250M graphics processor. Ubuntu 10.04 LTS was left at its defaults, including the EXT4 file-system. This notebook is on the completely different end of the hardware spectrum from the Atom computers that first illustrated this problem.