When completing these test results from the four most recent Ubuntu releases they certainly weren't what we had expected. In a number of these tests Ubuntu was slower with the newer releases on the same exact hardware and test configuration. We had even repeated a number of these tests again to verify our findings, and the results were the same. In addition, depending upon the test, the Phoronix Test Suite runs each test as much as five times before calculating the results. Below are some of our findings from the different test coverage areas.
OpenGL / Gaming: With one exception, the performance of OpenArena, World of Padman, and Unreal Tournament 2004 was similar between Ubuntu 7.04 "Feisty Fawn" and Ubuntu 8.10 "Intrepid Ibex". For these tests it's important to keep in mind that the closed-source ATI Catalyst driver was used since it wasn't until earlier this year that there was open-source 3D acceleration in the Mesa stack for the ATI Mobility Radeon X1400 (R500 series). Had we been able to use the Mesa and X.Org drivers with each Ubuntu release, we would have likely noticed improved performance over time considering the work that has gone on within these areas. Alternatively, there would have also been improvements to speak of had we used the ATI Catalyst drivers that were released around the same time of each distribution release. What is important to note is that the gaming performance hadn't dropped with the newer releases.
Desktop / GTK: When using GtkPerf to measure the performance of the GtkDrawingArea widget with Pixbufs, the performance was more or less identical with Ubuntu 7.04, 7.10, and 8.04. When switching to Ubuntu 8.10, however, the performance had dropped by more than 20%. This could be attributed to the binary-only ATI driver with X Server 1.5 / X.Org 7.4. The performance of 5,000 iterations of the GtkRadioButton widget in Ubuntu 8.10 was slower than Ubuntu 7.04, which could be caused by GTK changes or the ATI driver.
Memory: Our main system memory benchmark is RAMspeed and here the integer and floating-point memory performance was similar between the Ubuntu Gutsy Gibbon, Hardy Heron, and Intrepid Ibex releases, but was marginally slower than Ubuntu 7.04. In no RAMspeed test did the Gutsy Gibbon, Hardy Heron, or Intrepid Ibex releases out perform the Feisty Fawn. In the Bandwidth benchmark, we had experienced a similar situation with 7.10, 8.04, and 8.10 running slower than 7.04, but by a much greater margin. Here these three newer releases were almost 50% slower.
Audio/Video Encoding: In all of the audio encoding tests (MP3, Ogg, FLAC, and WavPack), Ubuntu 7.10 was the fastest at converting the WAV audio file to the different formats. Other common traits to the audio encoding tests were that Ubuntu 7.04 was the second fastest, but both Ubuntu 8.04 and 8.10 were by far the slowest at audio encoding. In all four of the audio tests, Ubuntu Hardy and Intrepid were nearly 50% slower or greater. As all of these audio tests are built from source, this slowdown may be attributed to a difference in default optimizations or Intel regressions within GCC 4.2/4.3. When it came to video encoding with FFmpeg to convert our sample AVI file to an NTSC VCD, the 7.10, 8.04, and 8.10 releases were also much slower than the GCC 4.1-based Ubuntu 7.04.
The conclusion is continued on the next page.