What's Going On With Iveland & OpenBenchmarking.org
Last month I said what OpenBenchmarking.org is and how it should change the benchmarking / automated testing landscape once it's released in conjunction with Phoronix test Suite 3.0 "Iveland" early next year. I have also showed off the new graphing capabilities for this software and provided another update at the end of last month. Here now is another update with some more exciting details.
OpenBenchmarking.org is an open, collaborative testing platform designed by Phoronix Media and the developers behind the Phoronix Test Suite, the premiere and most comprehensive benchmarking platform for Linux and other operating systems. OpenBenchmarking.org makes the Phoronix Test Suite an even more extensible platform for conducting automated tests with complete integration into Phoronix Test Suite 3.0 "Iveland" as well as within Phoromatic, an online test remote management system designed for managing test farms within enterprise environments.
The OpenBenchmarking.org infrastructure provides public and private storage of test result data (including system logs and benchmark outputs) and effective collaboration tools for sharing results and efficiently comparing multiple test result sets. OpenBenchmarking.org also provides a package management system for making accessible new, updated, and third-party test profiles / suites to the users of the Phoronix Test Suite. Other services provided by OpenBenchmarking.org include making public the standardized test profile specification implemented by the Phoronix Test Suite and OpenBenchmarking.org statistical data showcasing detailed software and hardware metrics under Linux.
First off, yesterday I talked about the Phoronix Media server upgrades that are coming. "While it's not even out publicly yet, I expect OpenBenchmarking.org to eventually become bigger -- both in terms of traffic and even greater industry importance -- than Phoronix.com itself. For this to occur, the Phoronix Media web infrastructure will begin its upgrade process as soon as this coming weekend or next week when US traffic volumes are down for Thanksgiving. As part of the first phase of the upgrade, the server hardware capabilities will be more than tripled and Terabytes of more bandwidth will be allocated." The first phase of this rollout is still on track to happen this weekend when all Phoronix web properties should get a nice boost. Except the estimates of being three times greater is inaccurate, as it is actually found to be about six times greater in terms of hardware performance and capabilities. This should allow a lot more in enabling some very rich and interactive features for OpenBenchmarking.org.
Besides the improved graphs and results visualizations that have already been showcased and will continue to mature and become far more interactive, another one of the improvements now visible with the latest Iveland code-base is an improved display for system information viewing. Like the graphs, these system information tables will also be embeddable on other web-sites via OpenBenchmarking.org. This has been a feature that has been requested for use on Phoronix.com as well, which will happen. Below are a few examples of the system tables in their current Git form.
All of these tables are automatically generated by pts-core and pts_Graph. The Phoronix Test Suite and Phodevi (the Phoronix Device Interface), of course, automatically gather the system information so these tables are supplied without any user interaction. It is just not the testing process with the Phoronix Test Suite that is fully automated!
The pts_Graph library attempts to automatically compact all identical software/hardware information shown within these tables. Like the graphs, these tables are also rendered as SVG (in supported browsers) and are not simply HTML tables (in this article they are just screenshots). Your feedback is always appreciated on this initial implementation such as how to make it more visually appealing or compact the size of these tables even more.
The Phoronix Test Suite is also becoming more intelligent to determine what specifically is being tested and changed between multiple test runs in a result file so that more or less information can be automatically reported on a particular system component/sub-system to provide the user with the most relevant information to their individual needs.
The Phoronix Test Suite 3.0 client now supports HTTPS for secure transfers of test results and system/hardware information. Registered users of OpenBenchmarking.org will automatically default to HTTPS on both the client and web interfaces, to protect any confidential tests or testing of unreleased hardware/software. There is also going to be separate support for encryption of test results and signing of test data to verify authenticity. Phoromatic will also support HTTPS with Phoronix Test Suite 3.0.