2017: Should Linux Benchmarking Still Be Mostly Done With Ubuntu?

[AdamOne]

What about using Ubuntu Gnome? That's the same desktop as Fedora and Debian have by default (right?) and I guess it's common with Arch users also etc.
But still the distro that Valve and most AAA games support for maximum compatibility.

its the same environment but the ubuntu developers use relatively old versions of any package compared to other worlds of distros. red hat is much better but still relies on upgrading your distro from like F23 to 24, and it is _not_ always a successful upgrade, far from stable enough.
relling distros dont have the problem of old package-versions, that factor is eliminated.

[duby229]

It doesn't happen as often I would like, but when I do sell a Linux desktop, It's Redhat Workstation. Maybe not relevant to gaming benchmarks, but most everything else. I'd be willing to bet there are more paid systems in the world with it than with Ubuntu.

[schmidtbag]

People seem to forget why benchmarks are done. If Michael wants to prove that maximum potential of a distro, then something like Gentoo is the clear winner. But if he just wants to show how things perform and have progressed in a way that is relatable to the widest audience possible, then distros like Mint are necessary to test.

When you look at the average Windows hardware review, you don't see the reviewer tweaking the system to maximize the performance, because that makes it an unrealistic scenario and there will always be something else that can be improved, so it generally becomes a waste of time.

That being said, if Michael doesn't spend the time to highly optimize a distro, he should use the most common distro that uses very modern software, and, is targeted by corporations. That usually results in Ubuntu, OpenSUSE, and maybe Fedora. Debian is too outdated.

[macemoneta]

When benchmarking, the workload being benchmarked needs to be realistic for a representative indication of performance. If you're benchmarking end-user desktops, then Ubuntu is fine for the system software. If you're benchmarking data center servers, then Red Hat would probably be a better choice.

[Kano]

Basically I do not like Unity, but for benchmarks the PPA with newer mesa and kernels are most likely pretty easy to use - I would use KDE instead. Basically you could use Debian too but there are no PPA for that. If you use Debian stable there are sometimes mesa backports, those would work with Intel as there are no LLVM updates required but most likely you would need to use an unstable/experimental mix to test current code. That required more knowledge in case of problems, so just keep your test boxes. But please test OpenGL vs. Vulkan in lower res if you want to compare both, 4k is certainly more likely GPU limited but to compare a different codepath most likely you need to use pretty low res - maybe even 720p if 1080p does not show too many differences.

[notanoob]

Unless a distribution with significant better performance is used (Clear Linux), I do not see a reason to change. SteamOS and Ubuntu are the target distros for all games on Steam, it makes no sense to use something else just because some people have a personal grudge against Cannonical.

Exactly this. I hate the direction cannonical has been going. But they are still the target platform for most game devs and users so it only makes sense to use them for benchmarks. If you want the peak performance that is possible use clear/arch/gentoo and see the difference every once and a while. Well maybe not gentoo because time constraints >.>.

[Kemosabe]

Ubuntu might be attractive for a certain kind of Linux users. However it has a closed upstream development and repeatedly goes different ways than the rest of the Linux ecosystem which renders Ubuntu not representative and a bad candidate for benchmarks. With respect to its market share it should be benchmarked in addition though.

[sarfarazahmad]

please use debian testing, i have tried out a lot of distributions over a good amount of time. Debian testing is what should everything else be compared against. I myself am using Arch for quite a while but i think that latest upstream software isn't always the best. Debian Testing has the right amount of stability, less bloat, considerably bug free and fast software. It may not be the very latest but its fast and good. Just look at Michael's past benchmarking result on this very site itself.

[mike44]

Ubuntu is still the easiest and most widely used especially for gaming. But why not get your visitors os and aside Ubuntu also benchmark the most used by your visitors.
Apart os also interesting would be desktops used. FPS comparisons on different os/desktops would be a nice read.

[kyrios]

Despite of its "low market shares", I personally think that Solus would be a good candidate to be added from time to times in the benchmarks because they use many optimizations (inspired from Clear Linux) than - to my knowledge - other distributions do not apply. Also they provide an optimized steam client (with up-to-date libraries) so this might eventually a more significant impact on some benchmarks.