Ubuntu 7.04 to 8.10 Benchmarks: Is Ubuntu Getting Slower?

With the release of Ubuntu 8.10 coming out later this week we decided to use this opportunity to explore how the performance of this desktop Linux operating system has evolved over the past few releases. We performed clean installations of Ubuntu 7.04, Ubuntu 7.10, Ubuntu 8.04, and Ubuntu 8.10 on a Lenovo ThinkPad T60 notebook and used the Phoronix Test Suite to run 35 tests on each release that covered nine different areas of the system. After spending well more than 100 hours running these tests, the results are now available and our findings may very well surprise you.

For our testing we had used the final Intel 32-bit releases of the four most recent Ubuntu releases except for Ubuntu 8.10 "Intrepid Ibex" where we used the Intrepid release candidate. We had installed each of these releases with the same set of options and identical configurations during testing. A clean installation of Ubuntu was done each time over the entire hard disk drive. Once each distribution was installed we simply setup the proprietary ATI graphics driver and then proceeded to install our selected tests using the Phoronix Test Suite. Following that (and after a reboot of course), we then carried out our testing. We had used Phoronix Test Suite 1.4.0 Beta 1 "Orkdal" and all of their settings and test parameters were left at their defaults.

For those not familiar with the Phoronix Test Suite, it's our GPLv3 licensed benchmarking software that's supported on Linux, OpenSolaris, FreeBSD, and Mac OS X. The Phoronix Test Suite provides a framework for carrying out both qualitative and quantitative tests with support for running in a fully autonomous fashion. Right now in Phoronix Test Suite Orkdal are 79 test profiles and over 30 test suites. The Phoronix Test Suite also has a global repository for viewing and comparing test results, detects installed hardware and software, provides integrated graphing support, and has many other features. With the testing, the Phoronix Test Suite takes care of the entire process from downloading and installing the tests, acquiring any needed test dependencies (on supported distributions), running the tests with specified settings (and most tests are run between three and five times), and parsing of the results. Modules can be written around the Phoronix Test Suite to extend this functionality even further.

To recap the major packages in each Ubuntu release, Ubuntu 7.04 had shipped with the Linux 2.6.20 kernel, X Server 7.2.0, GCC 4.1.2, and its Java environment was Build 1.6.0-b105. With Ubuntu 7.10 it uses the Linux 2.6.22 kernel, X Server 1.3.0, GCC 4.1.3, and libgcj 4.2.1. Ubuntu 8.04 LTS switched to the Linux 2.6.24 kernel, X Server 1.4.0.90 (1.4.1 pre-release), GCC 4.2.4, and OpenJDK 1.6.0_0-b11. Lastly, Ubuntu 8.10 uses the Linux 2.6.27 kernel, X Server 1.5.2, GCC 4.3.2, and IcedTea 6 1.3.1 (build 1.6.0_0-b12).

The Lenovo ThinkPad T60 is made up of an Intel Core Duo T2400 processor, Intel Mobile 945 + ICH7-M Chipset, 1GB of DDR2 system memory, an 80GB HTS541080G9SA00 hard drive, and an ATI Mobility Radeon X1400 128MB graphics processor. With the Mobility Radeon X1400 we had used the ATI Catalyst 8.10 driver from AMD's web-site except for when using Ubuntu 8.10 where we had used the Catalyst 8.10 (fglrx 8.54) driver that ships with Ubuntu as it has Linux 2.6.27 and X.Org 7.4 support. Compiz and Enhanced Intel SpeedStep Technology (EIST) were disabled during testing.

With all of that said, once each distribution was setup in the same way we proceeded to run the Phoronix Test Suite. The tests we used had included the BYTE Unix Benchmark, SciMark 2.0, SQLite, Tandem XML, eSpeak Speech Engine, timed Apache compilation, timed PHP compilation, timed ImageMagick compilation, Bonnie++, Flexible IO Tester, GnuPG, OpenSSL, LAME MP3 encoding, Ogg encoding, FLAC encoding, WavPack encoding, FFmpeg encoding, OpenArena, World of Padman, Unreal Tournament 2004 Demo, GtkPerf, Bork File Encrypter, Java SciMark 2.0, and RAMspeed. On the following pages are our benchmark results from these tests on Ubuntu 7.04, 7.10, 8.04, and 8.10.