1. Computers
  2. Display Drivers
  3. Graphics Cards
  4. Memory
  5. Motherboards
  6. Processors
  7. Software
  8. Storage
  9. Operating Systems

Facebook RSS Twitter Twitter Google Plus

Phoronix Test Suite

OpenBenchmarking Benchmarking Platform
Phoromatic Test Orchestration

How Linux Benchmarking Will Change With Iveland & OpenBenchmarking.org

Michael Larabel

Published on 20 October 2010
Written by Michael Larabel
Page 1 of 1 - 7 Comments

Phoronix Test Suite 3.0 (codenamed "Iveland") has been under heavy development for more than a month and there is still at least three more months left of work before this major release will be christened. Today though it is time to publicly share the first details (aside from those that learned about it in the Augustiner tent at Oktoberfest) for one of the new components to be making up a critical piece of the Phoronix Test Suite 3.0 platform: OpenBenchmarking.org.

As was shared in my Phoronix Test Suite 3.0 development notes in early September, "A brand new, entirely redesigned "Phoronix Global" replacement to http://global.phoronix.com/ will be coming in tandem with Phoronix Test Suite 3.0 as part of Iveland's developments." OpenBenchmarking.org is to serve as this replacement to Phoronix Global, but with many more features to far trump its current capabilities.

OpenBenchmarking.org is the next-generation Phoronix Global platform for providing public and private storage of test result data, an integrated packaging system for remotely fetching and managing new test profiles and suites that comply with the OpenBenchmarking.org test specification, and a robust set of web-based tools for providing effective collaboration and analytical capabilities of Phoronix Test Suite results.

Essentially, OpenBenchmarking.org is the rebuild of Phoronix Global with all of the features (plus many more) that we have desired from this system since before Phoronix Test Suite 1.0 was released. Phoronix Global will continue to live around for those running a pre-Iveland Phoronix Test Suite 1.0/2.0 release, but OpenBenchmarking.org will be the central resource for those running Phoronix Test Suite 3.0 and later. The features and capabilities of the OpenBenchmarking.org software are radically different and far more advanced than that of Phoronix Global with a complete rewrite of the code-base taking place.

With OpenBenchmarking.org building in a packaging system that is seamlessly integrated into the Phoronix Test Suite client, there will no longer be any suites or test profiles actually shipping with the Phoronix Test Suite; all tests and suites will all be living in the OpenBenchmarking.org cloud. When you go to install a new test or suite, the relevant data will be automatically pulled from the OpenBenchmarking.org servers. This new method for handling of test profiles and suites provides several benefits, including the ability to immediately pull a new test profile for a given software component without needing to upgrade the entire Phoronix Test Suite package (or wait around for a new Phoronix Test Suite release), the ability to maintain the same exact test profile versions when upgrading the Phoronix Test Suite itself (for those organizations that may "lock" themselves into a particular set of test profile versions they wish to maintain for an extended period of time), and for third-parties to easily publish their own tests and suites while making it just as easy for any other individual to install these custom/third-party packages.

All of the core Phoronix Global features will remain intact with OpenBenchmarking.org, including the ability to easily benchmark your system(s) against another user's by simply knowing the result ID number, the ability to upload your test data anonymously, etc. There will also finally be the introduction of "shopping-cart-like comparisons" of test results that can all be done through the OpenBenchmarking.org web interface in a similar way to how it is handled with Phoromatic. With OpenBenchmarking.org there will also be the capabilities to better exchange result data and information between this global repository and the Phoromatic remote test management system and Phoromatic Tracker for better tracking potential performance regressions in a software stack. There will also be features to better assist those that may create a test profile simply to quickly and easily exhibit a regression -- whether it is performance-related or a unit test -- in an automated manner for any piece of software.

As part of the OpenBenchmarking.org initiative with Iveland, there have also been resources invested into a set of policies to more prominently display a user's test results within the global repository and allow them to be compared to other users, while at the same time trying to automatically detect if the result data integrity was compromised via manual editing or falsified otherwise to try to report a better score for competitive reasons.

While Phoronix.com and the development of the Phoronix Test Suite are handled by separate but linked entities, of interest to many Phoronix.com readers (and the other publications that leverage this software) will be the embeddable result graphs to be found on OpenBenchmarking.org and the improvements to be found within Iveland's pts_Graph library. There has long been a desire by some to have various statistical values and other technical metrics shown within the Phoronix Test Suite result graphs found on Phoronix.com and within the Phoronix Test Suite results viewer and other places. This will happen with OpenBenchmarking.org as these embeddable graphs will be rendered in SVG / JavaScript within supported browsers (and automatically fallback to PNG/HTML in other browsers) to allow for exposing in real-time individual result values, system logs, and other data. These far more advanced graphs will also be found within the Phoronix Test Suite results viewer on the client side, which is also receiving other new features to be talked about later on in the Iveland development cycle. Any other individuals and publications can also use these OpenBenchmarking.org embeddable graphs, which should lead to the most transparent testing processes within the industry when paired with the Phoronix Test Suite client and other components.

As expressed earlier, due to these very radical changes with Phoronix Test Suite 3.0 and the introduction of OpenBenchmarking.org, Iveland will not be introduced this quarter but rather will formally enter a production state in the first quarter of 2011. This is the first time in the Phoronix Test Suite history that a major update has not come out on a quarterly basis. Alpha and beta releases, however, should begin by late November and continue through early next year. OpenBenchmarking.org is just one of several major features to be found in Phoronix Test Suite 3.0 "Iveland", so stay tuned for more articles as we talk about the different features to further drive automated, open-source, collaborative testing on Linux, Solaris, BSD, Mac OS X, and Windows platforms.

About The Author
Michael Larabel is the principal author of Phoronix.com and founded the web-site in 2004 with a focus on enriching the Linux hardware experience and being the largest web-site devoted to Linux hardware reviews, particularly for products relevant to Linux gamers and enthusiasts but also commonly reviewing servers/workstations and embedded Linux devices. Michael has written more than 10,000 articles covering the state of Linux hardware support, Linux performance, graphics hardware drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org automated testing software. He can be followed via and or contacted via .
Latest Linux News
  1. OPNsense 15.7 Released As Fork Of Pfsense
  2. The Less-Powerful Intel Compute Stick With Ubuntu Will Soon Ship
  3. Kodi 15.0 Release Candidate 1 Arrives
  4. Fedora 23: Python 3 Default Approved; Netizen Spin Rejected
  5. GNOME Shell & Mutter Just Landed More Wayland Improvements
  6. Ubuntu MATE Announces A Partnership With A PC Hardware Vendor
  7. Linux 4, GCC v. Clang & Vulkan Were Among The Hot Stories So Far This Year
  8. FUSE Starts Working On Scalability Improvements With Linux 4.2
  9. Steam Linux Usage Continued Falling In June
  10. Blender 2.75 Released With AMD OpenCL Support, Multi-View/Stereo 3D Pipeline
Latest Articles & Reviews
  1. 6-Way File-System Comparison On The Linux 4.1 Kernel
  2. How KDE VDG Is Trying To Make Open-Source Software Beautiful
  3. Attempting To Try Out BCache On The Linux 4.1 Kernel
  4. CompuLab's Fitlet Is A Very Tiny, Fanless, Linux PC With AMD A10 Micro
Most Viewed News This Week
  1. Kubuntu 15.10 Could Be The End Of The Road
  2. KDBUS Won't Be Pushed Until The Linux 4.3 Kernel
  3. Pinos Is For Linux Video What PulseAudio Is For Audio
  4. The State & Complications Of Porting The Unity Editor To Linux
  5. The Staging Pull For Linux 4.2: "Big, Really Big"
  6. Latest Rumor Pegs Microsoft Wanting To Buy AMD
  7. "PulseVideo" Coming To Complement PulseAudio?
  8. Exciting Features Merged So Far For The Linux 4.2 Kernel