1. Computers
  2. Display Drivers
  3. Graphics Cards
  4. Memory
  5. Motherboards
  6. Processors
  7. Software
  8. Storage
  9. Operating Systems


Facebook RSS Twitter Twitter Google Plus


Phoronix Test Suite

OpenBenchmarking.org

What Linux Distribution Should Be Benchmarked The Most?

Phoronix

Published on 26 July 2013 01:18 AM EDT
Written by Michael Larabel in Phoronix
223 Comments

Several Phoronix readers have brought up an important topic recently on Twitter and within our forums: what Linux distribution should really be be benchmarked the most? Ubuntu has traditionally been the most tested Linux platform here, but times may be changing.

As Ubuntu deviates more and more from the "conventional desktop Linux" stack with the continued evolution of Unity, the adoption of Mir over X.Org or Wayland, and other changes to distinguish Ubuntu from the hundreds of other Linux distributions, more readers are calling for Ubuntu not to be our default testing platform.

The reason for using Ubuntu, of course, has been that by Canonical's distribution is easily the most popular desktop Linux distribution. With it being the most widely used desktop Linux distribution, the results obviously apply to the greatest number of readers. This has been the primary reason for benchmarking with Ubuntu is that it gives the hardware reviews and benchmarking articles maximum relevance -- and thus page views.

As I have repeatedly explained in the forums and elsewhere, I have nothing against benchmarking other Linux distributions. It's commonly done at Phoronix in the context of server Linux distribution comparisons, our common multi-way Linux/BSD distribution comparisons, major releases of tier-one Linux distributions, etc. These multi-way Linux distributions shown at Phoronix have shown that most Linux distributions perform close to the same -- the common changes between distributions tend to be facial and not too performance sensitive, though with some exceptions. Additionally, with the latest Ubuntu packaging, the Unity+Compiz desktop configuration no longer has a big performance penalty like it once did compared to other desktop environments. Of course, XMir will change that (at least temporarily), but we're not quite there yet.

Prior to Ubuntu's rise, Fedora (Core) was my personal favorite Linux distribution and prior to that and the original Red Hat days, and prior to that I started off with Mandrake. The reason Fedora (and some other distributions) are not benchmarked more in the ever-changing Linux landscape is their use of debug builds by default for development packages, which can negatively affect performance. Around the time of a major stable release of Fedora, there's plenty of benchmarks on Phoronix with its bleeding edge but stable and non-debug state.

Besides the other mentioned distributions I am fond of, I also really like Xubuntu a lot. On my few personal systems not devoted to production purposes and/or benchmarking, Xfce is the most common desktop environment after the GNOME3 fallout. I've also enjoyed stints with Manjaro Linux, the defunct KateOS, Knoppix, openSUSE, CentOS, etc. Contrary to the belief of some Phoronix readers, I am not some "diehard Ubuntu fan boy" and I am also not some "Canonical hater." This is just like how Phoronix readers commonly go from calling me an AMD or NVIDIA "fan boy."

Aside from the debug build situation during the development cycle, another benefit of Ubuntu is their "daily mainline kernel PPA" where Debian packages are available daily (and tagged) releases of the mainline Linux kernel. When I maintain dozens of systems in my office/labs and am routinely reformatting them, it's very convenient to simply download the very latest Linux kernel code (important for Phoronix tests) without having to build the kernel from source. Beyond saving a few minutes each time, the mainline kernel PPA is very easy to cite Phoronix readers to when asking about how to install the very latest Linux kernel, what the kernel configuration was like as found in a Phoronix article, etc (enhancing the reproducibility of Phoronix tests). I'm not aware of any other Linux distribution offering easy, daily-updated kernel binary packages. On a similar note, LLVM/Clang has recently begun offering their own daily Ubuntu packages as well, which add convenience there.

Of course, once Mir/XMir is mainline in Ubuntu, many of our graphics driver articles and GPU reviews will likely be on other Linux distributions -- in either replacement of Ubuntu testing or as a complement depending upon the situation. Phoronix testing has already shown using XMir leads to performance penalties at present, plus the Mir EGL platform support isn't even mainline yet in Mesa. With being a very frequent Mesa Git user, I don't want to deal with having to patch the Mir support on Mesa each and every time plus having to deal with any breakage. But as long as Ubuntu remains very relevant on the Linux desktop, it will likely still be used a lot for other areas like Linux kernel and compiler testing where it's more of a virgin environment.

As expressed many times already in the forums and Twitter, why Arch Linux, Gentoo, and other popular rolling-release source distributions aren't used more is that it's hard to define a good reference point. With all of the test results shared on Phoronix I try to strive on reproducibility and accuracy, but with a rolling-release distribution and/or everything built from source, it's harder for others to exactly mirror my configurations than if I am using a binary distribution and only a few defined tweaks (if any). Plus with my many different systems and constantly switching out hardware and reformatting, I want something that's quick, easy, and reproducible during setup. With being responsible for nearly all of the content on Phoronix and seven days a week working on anywhere from 1~3 featured articles per day with benchmarks plus up to 12 or more news articles per day, on top of Phoronix Test Suite / OpenBenchmarking.org / Phoromatic programming, and other tasks every day, properly utilizing time is very important...unless you want Phoronix to be another Linux site to shutdown.

Anyhow, with those initial thoughts shared, I'm interested in hearing what other Phoronix readers would be interested in seeing used as the Linux platform for software testing, so please voice your opinion in the forums (use the comments link below) or let me know on Twitter via @MichaelLarabel. Thanks!

About The Author
Michael Larabel is the principal author of Phoronix.com and founded the web-site in 2004 with a focus on enriching the Linux hardware experience and being the largest web-site devoted to Linux hardware reviews, particularly for products relevant to Linux gamers and enthusiasts but also commonly reviewing servers/workstations and embedded Linux devices. Michael has written more than 10,000 articles covering the state of Linux hardware support, Linux performance, graphics hardware drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org automated testing software. He can be followed via and or contacted via .
Latest Linux Hardware Reviews
  1. Rosewill RS-MI-01: An Ultra Low-Cost Mini-ITX Chassis
  2. D-Link DCS-2330L HD Wireless Network Camera
  3. Gigabyte AM1M-S2H
  4. AMD's New Athlon/Semprons Give Old Phenom CPUs A Big Run For The Money
Latest Linux Articles
  1. The Performance Of Fedora 20 Updated
  2. Clang Fights GCC On AMD's Athlon AM1 APU With Jaguar Cores
  3. Ubuntu 14.04 LTS vs. Oracle Linux vs. CentOS vs. openSUSE
  4. How Much Video RAM Is Needed For Catalyst R3 Graphics?
Latest Linux News
  1. Red Hat Enterprise Linux 7.0 Is Looking Fantastic
  2. Intel Is Launching An Interesting Bay Trail NUC Next Week
  3. Another X.Org EVoC Proposed For OpenGL 4+ Tests
  4. The Best Features Coming With Qt 5.3
  5. Red Hat's RHEL7 RC ISO Is Now Publicly Available
  6. Nuclear Dawn Seems To Run Fine On AMD Linux
  7. KDE 4.14 Release Schedule Published
  8. GCC 4.9.0 Released, Brings Many Compiler Features
  9. OpenSSL Forked By OpenBSD Into LibreSSL
  10. GNOME Has Big Plans For Its Maps Application
  11. NVIDIA Will Soon Probably Introduce OpenCL 1.2 Linux Support
  12. Google Is Financing A Lot Of Great Open-Source Work This Summer
Latest Forum Discussions
  1. Announcing radeontop, a tool for viewing the GPU usage
  2. New card. Open source drivers only.
  3. The GNOME Foundation Is Running Short On Money
  4. Linux Kernel Developers Fed Up With Ridiculous Bugs In Systemd
  5. The Most Amazing OpenGL Tech Demo In 64kb
  6. Script for Fan Speed Control
  7. Torvalds Is Unconvinced By LTO'ing A Linux Kernel
  8. ReactOS Working On A Community Windows OS