New OpenBenchmarking.org Features Enhance Discovering Popular + Reliable Tests
On top of all the changes rolled out last month when the new OpenBenchmarking.org went live in alpha form, the latest batch of changes enabled are on the test profile viewing side:
On the test profile pages, all tests that have been around longer than a quarter now show a breakdown of their per-month statistics based on those opting to upload their results and those opting into the anonymous statistics/usage reporting. This allows seeing a per-month breakdown for the popularity of a given test in evaluating its popularity/relevance when choosing benchmarks to run with the Phoronix Test Suite.
For tests exposing multiple options when running the given test, the popularity of each of those options based on the gathered public results on OpenBenchmarking.org are presented. So for users not particularly familiar with a given test, these metrics can provide some guidance about what others are running as their selected option(s).
These metrics can be particularly helpful when running larger tests that may expose a lot of options.
Test profile pages also now expose much more detailed views on expected time requirements for not only the given test but depending upon what option(s) you select for a much more detailed look at time requirements rather than just the previous higher level look at the average run-time for a given test. This can help if pressed by time requirements or not looking to assemble a test suite that takes days / too much time to execute.
The other addition this weekend is also offering a more detailed look at the reliability/accuracy of a given test profile by analyzing the standard deviation across all given runs for the selected benchmark/options to see the average standard deviation based on all public data. This provides a look at whether a given test may be rather noisy depending upon the option(s) selected or at least based on all public benchmarks with it is known to deliver fairly solid numbers, etc.
These additions and other minor enhancements are now live via OpenBenchmarking.org for the nearly 600 different test profiles (593 at the moment). Speaking of test profiles, some new tests introduced over the past week include GPAW, MOCASSIN, LLNL's Kripke, Tencent's NCNN, Alibaba's MNN, Incompact3D, Hashcat, and RedShift, as well as several updates to existing test profiles.
The new OpenBenchmarking.org and Phoronix Test Suite 10.0 are currently on track for their official debut in October.