Making It Even Easier To Gauge Your System's Performance
Back during the slower periods of the winter holidays, I started working on some improvements coming up this year to the Phoronix Test Suite and OpenBenchmarking.org. One of the first initiatives was working on the infrastructure to allow for some big data processing to be punted off to spare benchmark systems locally rather than in the data center where the OpenBenchmarking.org / Phoronix Media public web infrastructure resides. That allows for still routine generating of the new analytics but much quicker and opening up to more possibilities with much more expansive computing resources locally than on the public web servers. The first of the features benefiting from that shift is on Phoronix Test Suite runs to provide via OpenBenchmarking.org an inline box plot showing all of the performance data for that particular test profile / test version / test configuration... That provides an immediate insight into how that particular system under test compares to the dozens / hundreds / thousands (depending upon the popularity of the test profile, how new of a test/software is being evaluated, and the configuration like screen resolution or intensity of settings) of other results available via OpenBenchmarking.org from other users' systems.
The box plot provides the interesting look at the distribution of results albeit condensed to fit within the constraints of a terminal and its coarseness but still indicating the common percentiles and median of all the collected public data on OpenBenchmarking.org. Overlaid onto that box plot of all the results for that particular test profile configuration is also the current result as well as prominent composite results... Those other results basically come down to comparable components that are popular / frequently benchmarked, first appeared on OpenBenchmarking.org within the past two years (i.e. not vintage hardware), and where there is enough confidence in the numbers to show its average representation. The display also indicates what percentile the current result would fit into compared to the community of data. The size of the sample set used is also indicated.
Well, it's probably easiest to explain just by looking at some of the screenshots in this article that were done using the latest Phoronix Test Suite Git.
Outliers are obviously dropped from those data-sets among other precautions when interpreting the mass collection of data on OpenBenchmarking.org to reduce the chance of gaming the system, etc.
These inline plots are shown automatically underneath the test result when there is enough data available from OpenBenchmarking.org for that particular test... And, of course, assuming you have an Internet connection present. Of course, the test result isn't transmitted to OpenBenchmarking.org at all unless you opt to upload your results at the end of the result file. The Phoronix Test Suite is simply querying matching data-sets based upon the test profile configuration hash. Unless it's a brand new test profile or one of the very niche tests, there is this data available.
This functionality is complementary to the seamless / dynamic OpenBenchmarking.org comparisons. But with that older feature, it's simply comparing the current result to an OpenBenchmarking.org-picked singular result file for comparison purposes. This new functionality provides a look on a more macro scale compared to all of the public matching benchmark results for a much broader look at how your system's performance compares.
Similar revised functionality is currently being worked on for the OpenBenchmarking.org website to offer much more detail than what can be condense into a few lines of the terminal window.
A overhaul to the actual OpenBenchmarking.org web interface and allowing much more data analytics to be exposed, etc, is on track for later this year by Phoronix Test Suite 9.0.
Of course, this is supported on all platforms where the Phoronix Test Suite functions just not Linux but also BSDs, Windows, macOS, and Solaris. And, yes, it works for any of the hundreds of different test profiles (benchmarks) assuming there is enough data to analyze.
Feedback is certainly welcome and this functionality will continue to be worked out ahead of the Phoronix Test Suite 8.6 stable release in February.
5 Comments
