The New OpenBenchmarking.org Now In Alpha For Better Hardware & Benchmark Discovery

As alluded to previously, a major overhaul of OpenBenchmarking.org has been in the works for a number of months now including a completely brand new analytics engine as part of the Phoronix Test Suite 10.0 development with its release due out later this year. With the new OpenBenchmarking.org now in good enough shape at least for the internal infrastructure, this new version is being opened up to the public today while over the weeks ahead more features will continue to be flipped on.

OpenBenchmarking.org continues to serve as the "cloud" to the Phoronix Test Suite for not only aggregating public benchmark results for those wanting to upload their test results for the world to see but also for serving as the "package manager" for serving up new/updated test profiles and test suites to the Phoronix Test Suite client. OpenBenchmarking.org also offers up a wealth of analytics and other features thanks to leveraging all of the Phoronix Test Suite generated data that users can opt in to sharing or comparing their own systems against other hardware around the world.

The user interface has been refreshed and will continue to be tweaked over the coming weeks. The web interface remains predominantly desktop focused given that is where most users will be diving into performance and benchmark data, but there have also been some mobile enhancements too. But for the time being the mobile version is less complete than the desktop side. Internally much of the OpenBenchmarking.org code has been rewritten including its new analytics engine that has been under development for about two years in being able to effectively analyze the immense amount of data collected by OpenBenchmarking.org over the past decade.

One of my personal favorite additions is that cumulative performance data is now easily shown on test profile pages where there is enough statistically significant data collected across multiple runs and other factors in order to provide an "overview" of the component under test for that given test profile's workload.

Previously users would have to search and find given components they are interested in and matching among the hundreds of different test profiles available. Now it's trivial to say find the fastest CPUs for GIMP or RawTherapee imaging, perhaps the best hardware for Blender, how fast different CPUs can compile the Linux kernel, how fast can CPUs decode AV1 content, and much more out of the hundreds of different tests available. On the test profile pages are these performance classifications not only for the latest test profile but also for prior versions of the tests and for different options the test profile may expose to the user.

For new or less popular test profiles where there is not enough public data to be able to offer such performance metrics breakdowns in a statistically significant manner, there is at least a histogram provided showing the distribution of the public results so users can have a rough gauge of performance expectations when comparing the results of their own systems.

Another area seeing a lot of work has been on the OpenBenchmarking.org search area. There is better detection/recognition of computers components and in cases of like CPUs will now display additional information that can be automatically detected based upon the collected data.

Also in the case of searching a CPU it more prominently displays the /proc/cpuinfo and lscpu outputs when available rather than users having to click through result files looking for such outputs. Similarly, when searching for a motherboard is also the lspci output for seeing the onboard components and utilized drivers.

The search result page will also show some of the performance classifications of how that given component performs in various test profiles in cases where there is enough information accumulated for being able to provide an accurate assessment in the relevant test profiles.

Beyond the improved search capabilities, there are also new pages providing breakdown listings of processors, storage devices, and more.

The test/benchmark listings and suites page also has improved searching and sorting capabilities as well as showing tests/suites that may be new, the most downloaded, or other attributes.

When it comes to the primary focus of OpenBenchmarking.org for viewing actual benchmark results from users, that area has been improved too. In fact, OpenBenchmarking.org and the Phoronix Test Suite local result viewer are finally using the same shared code path and in turn will see all the same features supported moving forward and with a similar UI.

The improved OpenBenchmarking.org result viewer now offers many more options previously only exposed through the local result viewer, makes it easier to generate performance-per-dollar metrics, more easily highlight and hide results, and much more. There are also improvements when exporting results to TXT/PDF/CSV/etc and improved sorting and other means of slicing and dicing the data.

There are also other extras like when detecting a CPU comparison in a given result file, extra graphs are generated on the tabbed display for showing the performance-per-core/thread and approximate performance-per-clock, etc. Plus there are a lot of other smaller refinements throughout.

For corporations/organizations carrying out internal/private testing, the new OpenBenchmarking.org also allows users to opt-in their IP address to the result upload firewall. As yet another safeguard for helping companies using the Phoronix Test Suite for carrying out confidential testing, users can easily add IP addresses to the result upload firewall on OpenBenchmarking.org. Whether accidentally or otherwise, should an upload be attempted from a given IP address, it will be rejected and the supplied email address will be notified of the incident. Users on those IP address(es) can still access OpenBenchmarking.org though as well as obtaining new tests/suites and the other services offered by this open-source benchmarking cloud platform.

Visiting OpenBenchmarking.org should now be routing you to the new version. Keep in mind the new OpenBenchmarking.org is currently in alpha with more improvements and features coming in the weeks ahead prior to the formal launch with the Phoronix Test Suite 10.0 next quarter. Bug reports, feature requests, etc, are welcome and appreciated.