1. Computers
  2. Display Drivers
  3. Graphics Cards
  4. Memory
  5. Motherboards
  6. Processors
  7. Software
  8. Storage
  9. Operating Systems


Facebook RSS Twitter Twitter Google Plus


Phoronix Test Suite

OpenBenchmarking.org

What Linux Distribution Should Be Benchmarked The Most?

Phoronix

Published on 26 July 2013 01:18 AM EDT
Written by Michael Larabel in Phoronix
226 Comments

Several Phoronix readers have brought up an important topic recently on Twitter and within our forums: what Linux distribution should really be be benchmarked the most? Ubuntu has traditionally been the most tested Linux platform here, but times may be changing.

As Ubuntu deviates more and more from the "conventional desktop Linux" stack with the continued evolution of Unity, the adoption of Mir over X.Org or Wayland, and other changes to distinguish Ubuntu from the hundreds of other Linux distributions, more readers are calling for Ubuntu not to be our default testing platform.

The reason for using Ubuntu, of course, has been that by Canonical's distribution is easily the most popular desktop Linux distribution. With it being the most widely used desktop Linux distribution, the results obviously apply to the greatest number of readers. This has been the primary reason for benchmarking with Ubuntu is that it gives the hardware reviews and benchmarking articles maximum relevance -- and thus page views.

As I have repeatedly explained in the forums and elsewhere, I have nothing against benchmarking other Linux distributions. It's commonly done at Phoronix in the context of server Linux distribution comparisons, our common multi-way Linux/BSD distribution comparisons, major releases of tier-one Linux distributions, etc. These multi-way Linux distributions shown at Phoronix have shown that most Linux distributions perform close to the same -- the common changes between distributions tend to be facial and not too performance sensitive, though with some exceptions. Additionally, with the latest Ubuntu packaging, the Unity+Compiz desktop configuration no longer has a big performance penalty like it once did compared to other desktop environments. Of course, XMir will change that (at least temporarily), but we're not quite there yet.

Prior to Ubuntu's rise, Fedora (Core) was my personal favorite Linux distribution and prior to that and the original Red Hat days, and prior to that I started off with Mandrake. The reason Fedora (and some other distributions) are not benchmarked more in the ever-changing Linux landscape is their use of debug builds by default for development packages, which can negatively affect performance. Around the time of a major stable release of Fedora, there's plenty of benchmarks on Phoronix with its bleeding edge but stable and non-debug state.

Besides the other mentioned distributions I am fond of, I also really like Xubuntu a lot. On my few personal systems not devoted to production purposes and/or benchmarking, Xfce is the most common desktop environment after the GNOME3 fallout. I've also enjoyed stints with Manjaro Linux, the defunct KateOS, Knoppix, openSUSE, CentOS, etc. Contrary to the belief of some Phoronix readers, I am not some "diehard Ubuntu fan boy" and I am also not some "Canonical hater." This is just like how Phoronix readers commonly go from calling me an AMD or NVIDIA "fan boy."

Aside from the debug build situation during the development cycle, another benefit of Ubuntu is their "daily mainline kernel PPA" where Debian packages are available daily (and tagged) releases of the mainline Linux kernel. When I maintain dozens of systems in my office/labs and am routinely reformatting them, it's very convenient to simply download the very latest Linux kernel code (important for Phoronix tests) without having to build the kernel from source. Beyond saving a few minutes each time, the mainline kernel PPA is very easy to cite Phoronix readers to when asking about how to install the very latest Linux kernel, what the kernel configuration was like as found in a Phoronix article, etc (enhancing the reproducibility of Phoronix tests). I'm not aware of any other Linux distribution offering easy, daily-updated kernel binary packages. On a similar note, LLVM/Clang has recently begun offering their own daily Ubuntu packages as well, which add convenience there.

Of course, once Mir/XMir is mainline in Ubuntu, many of our graphics driver articles and GPU reviews will likely be on other Linux distributions -- in either replacement of Ubuntu testing or as a complement depending upon the situation. Phoronix testing has already shown using XMir leads to performance penalties at present, plus the Mir EGL platform support isn't even mainline yet in Mesa. With being a very frequent Mesa Git user, I don't want to deal with having to patch the Mir support on Mesa each and every time plus having to deal with any breakage. But as long as Ubuntu remains very relevant on the Linux desktop, it will likely still be used a lot for other areas like Linux kernel and compiler testing where it's more of a virgin environment.

As expressed many times already in the forums and Twitter, why Arch Linux, Gentoo, and other popular rolling-release source distributions aren't used more is that it's hard to define a good reference point. With all of the test results shared on Phoronix I try to strive on reproducibility and accuracy, but with a rolling-release distribution and/or everything built from source, it's harder for others to exactly mirror my configurations than if I am using a binary distribution and only a few defined tweaks (if any). Plus with my many different systems and constantly switching out hardware and reformatting, I want something that's quick, easy, and reproducible during setup. With being responsible for nearly all of the content on Phoronix and seven days a week working on anywhere from 1~3 featured articles per day with benchmarks plus up to 12 or more news articles per day, on top of Phoronix Test Suite / OpenBenchmarking.org / Phoromatic programming, and other tasks every day, properly utilizing time is very important...unless you want Phoronix to be another Linux site to shutdown.

Anyhow, with those initial thoughts shared, I'm interested in hearing what other Phoronix readers would be interested in seeing used as the Linux platform for software testing, so please voice your opinion in the forums (use the comments link below) or let me know on Twitter via @MichaelLarabel. Thanks!

About The Author
Michael Larabel is the principal author of Phoronix.com and founded the web-site in 2004 with a focus on enriching the Linux hardware experience and being the largest web-site devoted to Linux hardware reviews, particularly for products relevant to Linux gamers and enthusiasts but also commonly reviewing servers/workstations and embedded Linux devices. Michael has written more than 10,000 articles covering the state of Linux hardware support, Linux performance, graphics hardware drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org automated testing software. He can be followed via and or contacted via .
Latest Linux Hardware Reviews
  1. The MSI X99S SLI PLUS Is Working & Running Well On Linux
  2. NVIDIA GeForce GTX 980: The Best GPU For Linux Gamers
  3. ROCCAT LUA: A Linux-Friendly Gaming Mouse
  4. Cheetah Mounts: The Affordable Way To Put Your TV On The Wall
Latest Linux Articles
  1. Preview: Radeon Gallium3D Performance For CS:GO On Linux
  2. XWayland Linux Gaming Performance With GNOME Wayland On Fedora 21
  3. EXT4/Btrfs/XFS/F2FS Benchmarks On Linux 3.17
  4. Fedora 21 Alpha First Impressions: It's Great
Latest Linux News
  1. Radeon DRM Gets New Information Ioctl Queries
  2. Mir 0.8 Works On Less ABI Breakage, Touchspots, Responsiveness
  3. CS:GO For Linux Gains Better Stability, Community Server Support
  4. NVIDIA Issues Updated 340.46 Long-Lived Driver Release
  5. KDE Plasma 5.1 Now In Beta
  6. Systemd & Debian Were Most Popular In September
  7. Microsoft Announces... Windows 10 With A Start Menu
  8. Borderlands 2 Launches On Steam For Linux
  9. Debian Jessie Might Get Rid Of The kFreeBSD Port
  10. Fedora Might Try A New Scheduling Strategy For Its Releases
Latest Forum Discussions
  1. Microsoft Announces... Windows 10 With A Start Menu
  2. NVIDIA Alerts Nouveau: They're Starting To Sign/Validate GPU Firmware Images
  3. Updated and Optimized Ubuntu Free Graphics Drivers
  4. Nero CD/DVD Burning Software On Linux Is Dead
  5. New Group Calls For Boycotting Systemd
  6. FSF Issues Statement On Shellshock Bash Vulnerability
  7. X.Org Women Outreach Program Only Turns Up Two Applicants So Far
  8. X.Org Is Looking For Some Female Help