1. Computers
  2. Display Drivers
  3. Graphics Cards
  4. Memory
  5. Motherboards
  6. Processors
  7. Software
  8. Storage
  9. Operating Systems


Facebook RSS Twitter Twitter Google Plus


Phoronix Test Suite

OpenBenchmarking.org

What Linux Distribution Should Be Benchmarked The Most?

Phoronix

Published on 26 July 2013 01:18 AM EDT
Written by Michael Larabel in Phoronix
226 Comments

Several Phoronix readers have brought up an important topic recently on Twitter and within our forums: what Linux distribution should really be be benchmarked the most? Ubuntu has traditionally been the most tested Linux platform here, but times may be changing.

As Ubuntu deviates more and more from the "conventional desktop Linux" stack with the continued evolution of Unity, the adoption of Mir over X.Org or Wayland, and other changes to distinguish Ubuntu from the hundreds of other Linux distributions, more readers are calling for Ubuntu not to be our default testing platform.

The reason for using Ubuntu, of course, has been that by Canonical's distribution is easily the most popular desktop Linux distribution. With it being the most widely used desktop Linux distribution, the results obviously apply to the greatest number of readers. This has been the primary reason for benchmarking with Ubuntu is that it gives the hardware reviews and benchmarking articles maximum relevance -- and thus page views.

As I have repeatedly explained in the forums and elsewhere, I have nothing against benchmarking other Linux distributions. It's commonly done at Phoronix in the context of server Linux distribution comparisons, our common multi-way Linux/BSD distribution comparisons, major releases of tier-one Linux distributions, etc. These multi-way Linux distributions shown at Phoronix have shown that most Linux distributions perform close to the same -- the common changes between distributions tend to be facial and not too performance sensitive, though with some exceptions. Additionally, with the latest Ubuntu packaging, the Unity+Compiz desktop configuration no longer has a big performance penalty like it once did compared to other desktop environments. Of course, XMir will change that (at least temporarily), but we're not quite there yet.

Prior to Ubuntu's rise, Fedora (Core) was my personal favorite Linux distribution and prior to that and the original Red Hat days, and prior to that I started off with Mandrake. The reason Fedora (and some other distributions) are not benchmarked more in the ever-changing Linux landscape is their use of debug builds by default for development packages, which can negatively affect performance. Around the time of a major stable release of Fedora, there's plenty of benchmarks on Phoronix with its bleeding edge but stable and non-debug state.

Besides the other mentioned distributions I am fond of, I also really like Xubuntu a lot. On my few personal systems not devoted to production purposes and/or benchmarking, Xfce is the most common desktop environment after the GNOME3 fallout. I've also enjoyed stints with Manjaro Linux, the defunct KateOS, Knoppix, openSUSE, CentOS, etc. Contrary to the belief of some Phoronix readers, I am not some "diehard Ubuntu fan boy" and I am also not some "Canonical hater." This is just like how Phoronix readers commonly go from calling me an AMD or NVIDIA "fan boy."

Aside from the debug build situation during the development cycle, another benefit of Ubuntu is their "daily mainline kernel PPA" where Debian packages are available daily (and tagged) releases of the mainline Linux kernel. When I maintain dozens of systems in my office/labs and am routinely reformatting them, it's very convenient to simply download the very latest Linux kernel code (important for Phoronix tests) without having to build the kernel from source. Beyond saving a few minutes each time, the mainline kernel PPA is very easy to cite Phoronix readers to when asking about how to install the very latest Linux kernel, what the kernel configuration was like as found in a Phoronix article, etc (enhancing the reproducibility of Phoronix tests). I'm not aware of any other Linux distribution offering easy, daily-updated kernel binary packages. On a similar note, LLVM/Clang has recently begun offering their own daily Ubuntu packages as well, which add convenience there.

Of course, once Mir/XMir is mainline in Ubuntu, many of our graphics driver articles and GPU reviews will likely be on other Linux distributions -- in either replacement of Ubuntu testing or as a complement depending upon the situation. Phoronix testing has already shown using XMir leads to performance penalties at present, plus the Mir EGL platform support isn't even mainline yet in Mesa. With being a very frequent Mesa Git user, I don't want to deal with having to patch the Mir support on Mesa each and every time plus having to deal with any breakage. But as long as Ubuntu remains very relevant on the Linux desktop, it will likely still be used a lot for other areas like Linux kernel and compiler testing where it's more of a virgin environment.

As expressed many times already in the forums and Twitter, why Arch Linux, Gentoo, and other popular rolling-release source distributions aren't used more is that it's hard to define a good reference point. With all of the test results shared on Phoronix I try to strive on reproducibility and accuracy, but with a rolling-release distribution and/or everything built from source, it's harder for others to exactly mirror my configurations than if I am using a binary distribution and only a few defined tweaks (if any). Plus with my many different systems and constantly switching out hardware and reformatting, I want something that's quick, easy, and reproducible during setup. With being responsible for nearly all of the content on Phoronix and seven days a week working on anywhere from 1~3 featured articles per day with benchmarks plus up to 12 or more news articles per day, on top of Phoronix Test Suite / OpenBenchmarking.org / Phoromatic programming, and other tasks every day, properly utilizing time is very important...unless you want Phoronix to be another Linux site to shutdown.

Anyhow, with those initial thoughts shared, I'm interested in hearing what other Phoronix readers would be interested in seeing used as the Linux platform for software testing, so please voice your opinion in the forums (use the comments link below) or let me know on Twitter via @MichaelLarabel. Thanks!

About The Author
Michael Larabel is the principal author of Phoronix.com and founded the web-site in 2004 with a focus on enriching the Linux hardware experience and being the largest web-site devoted to Linux hardware reviews, particularly for products relevant to Linux gamers and enthusiasts but also commonly reviewing servers/workstations and embedded Linux devices. Michael has written more than 10,000 articles covering the state of Linux hardware support, Linux performance, graphics hardware drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org automated testing software. He can be followed via and or contacted via .
Latest Linux Hardware Reviews
  1. Intel Xeon E5-1680 v3 & E5-2687W v3 Compared To The Core i7 5960X On Linux
  2. Intel 120GB 530 Series SSD Linux Performance
  3. Btrfs/EXT4/XFS/F2FS RAID 0/1/5/6/10 Linux Benchmarks On Four SSDs
  4. AMD's Windows Catalyst Driver Remains Largely Faster Than Linux Drivers
Latest Linux Articles
  1. Mesa 10.5-devel Brings Some Intel Haswell HD Graphics Changes Over Mesa 10.3
  2. NVIDIA vs. Nouveau Drivers With Linux 3.18 + Mesa 10.4-devel
  3. Is The Open-Source NVIDIA Driver Fast Enough For Steam On Linux Gaming?
  4. Linux 3.18 File-System Performance Minimally Changed But Possible Regressions
Latest Linux News
  1. Amazon's Fire TV Stick: A Nice, Affordable Media Center Option
  2. Google Puts Chrome NPAPI Support On Final Countdown
  3. There's New In-Fighting Over The Future Of Compiz
  4. GTK+ Inspector Gains More Features Ahead Of GNOME 3.16
  5. Clang 3.6 Will Hopefully Have OpenMP Support
  6. A Go Front-End Could Soon Be Landing In LLVM
  7. Linux 3.18-rc6 Released, A Worrisome Regression Remains
  8. HandBrake 0.10 Brings H.265 & VP8 Encoders
  9. Gngr: A New Web Browser Focused On Privacy
  10. Linux 3.18 Kernel: Not Much Change With Intel Haswell Performance
Latest Forum Discussions
  1. Hurrican SDL Port
  2. Roadmap to Catalyst 14.10 ?
  3. how to configure module phoromatic ?
  4. PulseAudio 6.0 Is Coming & Other Linux Audio Plans For The Future
  5. Debian Developer Resigns From The Systemd Maintainership Team
  6. Updated and Optimized Ubuntu Free Graphics Drivers
  7. Cant get working Kaveri APU - A10-7850k
  8. Script for Fan Speed Control