RFC: Seamless OpenBenchmarking.org Comparisons For The Phoronix Test Suite
Written by Michael Larabel in Phoronix Test Suite on 16 August 2017 at 06:16 AM EDT. 2 Comments
This week another new feature has landed in Phoronix Test Suite 7.4 Git for making it even easier for new (and existing) users of the Phoronix Test Suite to easily add additional perspective to their system's performance with OpenBenchmarking.org seamless comparisons.

With the existing Phoronix Test Suite workflow, when you would be running a benchmark you wouldn't see how your performance compares to others unless you were first running a comparison against an existing result file or otherwise until after the tests have completed and you have uploaded your results to OpenBenchmarking.org, where you can then easily compare that result file against other compatible results. With the latest Phoronix Test Suite development code, it provides seamless integration with the Phoronix Test Suite to show you a real-time comparison of your current test result against automatically-selected interesting OpenBenchmarking.org results.

Built off the infrastructure for Phoronix Test Suite 7.4's progressive result reporting now has the Phoronix Test Suite querying a new OpenBenchmarking.org API to obtain automatically interesting results that are comparable and shows how your own system stacks up to those. Even if you are not saving your results, this feature will still show you these matches/comparisons.

Your results and system information are still kept locally and not automatically uploaded to OpenBenchmarking.org unless you opt so at the end of the benchmarking process, as has always been the case. With this feature, the Phoronix Test Suite only exposes to OpenBenchmarking.org the test hash being run (the unique identifier based on the test profile / test version / test parameters, for ensuring a comparable result match) and the type of system being run (whether PTS has identified the system as embedded / cloud / server / workstation / desktop -- since if you are running benchmarks on a Raspberry Pi, you probably don't want to see performance results from Amazon EC2 or a Xeon workstation...). OpenBenchmarking.org will then automatically determine the best batch of results to send to the client for display alongside your current test result.

This is only for displaying in the CLI output and isn't automatically saving these comparison results or adding it to your result file, etc. Basically for quick display purposes if wanting to see how a given benchmark's performance on your system compares to others. So with a command as simple as phoronix-test-suite benchmark encode-mp3, you can see how your own system compares to others without even needing to save the results or upload your results to the Internet. Also displayed alongside this dynamic comparison is an OpenBenchmarking.org URL if wanting to learn more about those other systems under test.

This feature is automatically enabled if running with a fast Internet connection, if you are not running in a batch/auto-mode (such as via batch-run or triggered via Phoromatic), and if you are not already running your own comparison against saved results in which case you are already likely seeing the comparison data you want.

Anyhow, that's the basics of this newest feature to OpenBenchmarking.org and to leverage the mass amounts of data already on OpenBenchmarking.org. I will still be tweaking OB's selection algorithm and such for matching to the "best" / "most interesting" results still in the days ahead, but if anyone has any comments or feedback on this feature, feel free to contact us or comment in the forums. (And if any PTS user prefers not seeing the data, they can phoronix-test-suite unload-module ob_auto_compare, as the functionality is just implemented as a new module.)
About The Author
Author picture

Michael Larabel is the principal author of Phoronix.com and founded the site in 2004 with a focus on enriching the Linux hardware experience. Michael has written more than 10,000 articles covering the state of Linux hardware support, Linux performance, graphics drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org automated benchmarking software. He can be followed via Twitter or contacted via MichaelLarabel.com.

Related Phoronix Test Suite News
Popular News This Week