No announcement yet.

2017: Should Linux Benchmarking Still Be Mostly Done With Ubuntu?

  • Filter
  • Time
  • Show
Clear All
new posts

  • #81
    I'm late to the party as usual... I'd say testing the latest Ubuntu AND Fedora would provide the best overall coverage for most. My particular interest is servers, but my primary personal desktop right now is Fedora. But in the past I've often recommended Ubuntu to others. Depending on what you're measuring, results can diverge: like Ubuntu using and Fedora with Wayland. Of course if the particular benchmark wouldn't show a significant difference, I can't see any problem with publishing only the Ubuntu results.


    • #82
      Interesting discussion overall, with some good points all around.

      I'm thinking that indeed it makes the most sense to select a distribution based on what you're benchmarking. If you're benchmarking games, use SteamOS or Gears on Gallium. If you're benchmarking new hardware, use Arch or Tumbleweed. If you're benchmarking compiler/filesystem settings... Hm, well, that's harder to decide. Who is the target audience? Probably not regular users, and Arch makes the most sense then; maybe Fedora and Clear Linux. Same with compute: it will be mostly used on CentOS, but you hardly ever want to benchmark old versions, so Fedora makes the most sense. If you're benchmarking desktop environments, Ubuntu makes the most sense, since only it has (actual) support for Unity. If you're benchmarking the evolution of performance over time, some Ubuntu spin (Linux Mint perhaps) makes the most sense, since it has LTS releases that are still supported and a semiannual release cycle.

      It's true that for reproducibility, rolling releases may not be optimal, but then PTS already reports all the versions of relevant software. The next best thing would be a distro with a fast release cycle and new packages; Fedora, maybe.

      Overall there is not a whole lot of point in benchmarking Ubuntu by default, since it's not really the target audience for most of the benchmarks. It seems that Michael likes to use the PPA system for testing latest Mesa, but then Gears on Gallium/Tumbleweed is a better point of reference. Of course, for the most part there is not a whole lot of difference in performance anyway.