What Linux Benchmarks Would You Like To See Next?
Written by Michael Larabel in Phoronix on 23 October 2014 at 05:36 AM EDT. 56 Comments
PHORONIX --
At Phoronix.com and with the Phoronix Test Suite / OpenBenchmarking.org we're always looking to cater to the interests of more parties and as such are interested to see what other benchmarks you'd like to see incorporated.

With the Phoronix Test Suite and through OpenBenchmarking.org are hundreds of test profiles and over one hundred test suites. The latest versions of the most common (but not all) tests can be seen via this test profile listing page. Many of them are frequently featured in Phoronix.com hardware reviews, driver analysis stories, and other articles.

Beyond that, I'm always looking to incorporate new tests and see other options added -- not only for the benefit of Phoronix readers but also for the hundreds of other organizations relying upon the Phoronix Test Suite day-in and day-out for benchmarking. New test profiles don't benefit just us but also those organizations relying upon our open-source (GPLv3) testing software for conducting Linux hardware validation, driver regression testing, etc. There's thousands of independent users running the Phoronix Test Suite daily as well. As such, I'm curious about any other new tests that users would like to see incorporated that might have went under my radar in recent months.

In regards to requirements for making a good test profile, most simply, it should be some real-world workload not covered by current tests that can be easily automated from start to finish and produce a reproducible result. While open-source benchmarks are obviously preferred, closed-source benchmarks are supported too and have been on numerous instances well endorsed for being good quality tests.

The problem though is with many modern Steam Linux games not being good test candidates. There's been numerous complexities in being able to reliably and autonomously run Steam Linux gaming tests, among other issues, which is why those for instance have been limited via the phoronix-test-suite client. I've covered a bit of the story within Running Benchmarks On Other New Linux Games, among other articles. Many scientific workloads make great test cases too, but if they run too long on modern desktop hardware or require extensive dependency chains, many users lose interest in running them, which then defeats many of the benefits of having them incorporated into the Phoronix test Suite.

Anyhow, it's worth another routine shout-out for any other interesting test cases that may have cropped up and be suitable for new Linux/open-source benchmarking. New test cases can be easily added if they're easy to incorporate while program customizations and other changes may require commercial engagements from interested parties. A partial list of existing tests interested in for possible inclusion have been listed via the phoronix-test-suite TODO.

Test profiles themselves tend to be rather trivial if the upstream program can produce reliable, automated results. Among the examples of good "raw" test profiles include C-Ray and x264, among others. It's quite easy to get involved and many have come up with their own test profiles after running PTS themselves and checking out the ~/.phoronix-test-suite/test-profiles/ contents. Contributions are always appreciated and compensation for creating interesting test cases are always an option too.

So if you have any interesting test/benchmark requests or other feedback, feel free to let us know. If you'd like to simply support this open-source benchmarking work, please subscribe to Phoronix Premium or consider a PayPal contribution.
Related News
About The Author
Author picture

Michael Larabel is the principal author of Phoronix.com and founded the site in 2004 with a focus on enriching the Linux hardware experience. Michael has written more than 20,000 articles covering the state of Linux hardware support, Linux performance, graphics drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org automated benchmarking software. He can be followed via Twitter or contacted via MichaelLarabel.com.

Popular News This Week