1. Computers
  2. Display Drivers
  3. Graphics Cards
  4. Memory
  5. Motherboards
  6. Processors
  7. Software
  8. Storage
  9. Operating Systems


Facebook RSS Twitter Twitter Google Plus


Phoronix Test Suite

OpenBenchmarking.org

How Linux Benchmarking Will Change With Iveland & OpenBenchmarking.org

Michael Larabel

Published on 20 October 2010
Written by Michael Larabel
Page 1 of 1 - 7 Comments

Phoronix Test Suite 3.0 (codenamed "Iveland") has been under heavy development for more than a month and there is still at least three more months left of work before this major release will be christened. Today though it is time to publicly share the first details (aside from those that learned about it in the Augustiner tent at Oktoberfest) for one of the new components to be making up a critical piece of the Phoronix Test Suite 3.0 platform: OpenBenchmarking.org.

As was shared in my Phoronix Test Suite 3.0 development notes in early September, "A brand new, entirely redesigned "Phoronix Global" replacement to http://global.phoronix.com/ will be coming in tandem with Phoronix Test Suite 3.0 as part of Iveland's developments." OpenBenchmarking.org is to serve as this replacement to Phoronix Global, but with many more features to far trump its current capabilities.

OpenBenchmarking.org is the next-generation Phoronix Global platform for providing public and private storage of test result data, an integrated packaging system for remotely fetching and managing new test profiles and suites that comply with the OpenBenchmarking.org test specification, and a robust set of web-based tools for providing effective collaboration and analytical capabilities of Phoronix Test Suite results.

Essentially, OpenBenchmarking.org is the rebuild of Phoronix Global with all of the features (plus many more) that we have desired from this system since before Phoronix Test Suite 1.0 was released. Phoronix Global will continue to live around for those running a pre-Iveland Phoronix Test Suite 1.0/2.0 release, but OpenBenchmarking.org will be the central resource for those running Phoronix Test Suite 3.0 and later. The features and capabilities of the OpenBenchmarking.org software are radically different and far more advanced than that of Phoronix Global with a complete rewrite of the code-base taking place.

With OpenBenchmarking.org building in a packaging system that is seamlessly integrated into the Phoronix Test Suite client, there will no longer be any suites or test profiles actually shipping with the Phoronix Test Suite; all tests and suites will all be living in the OpenBenchmarking.org cloud. When you go to install a new test or suite, the relevant data will be automatically pulled from the OpenBenchmarking.org servers. This new method for handling of test profiles and suites provides several benefits, including the ability to immediately pull a new test profile for a given software component without needing to upgrade the entire Phoronix Test Suite package (or wait around for a new Phoronix Test Suite release), the ability to maintain the same exact test profile versions when upgrading the Phoronix Test Suite itself (for those organizations that may "lock" themselves into a particular set of test profile versions they wish to maintain for an extended period of time), and for third-parties to easily publish their own tests and suites while making it just as easy for any other individual to install these custom/third-party packages.

All of the core Phoronix Global features will remain intact with OpenBenchmarking.org, including the ability to easily benchmark your system(s) against another user's by simply knowing the result ID number, the ability to upload your test data anonymously, etc. There will also finally be the introduction of "shopping-cart-like comparisons" of test results that can all be done through the OpenBenchmarking.org web interface in a similar way to how it is handled with Phoromatic. With OpenBenchmarking.org there will also be the capabilities to better exchange result data and information between this global repository and the Phoromatic remote test management system and Phoromatic Tracker for better tracking potential performance regressions in a software stack. There will also be features to better assist those that may create a test profile simply to quickly and easily exhibit a regression -- whether it is performance-related or a unit test -- in an automated manner for any piece of software.

As part of the OpenBenchmarking.org initiative with Iveland, there have also been resources invested into a set of policies to more prominently display a user's test results within the global repository and allow them to be compared to other users, while at the same time trying to automatically detect if the result data integrity was compromised via manual editing or falsified otherwise to try to report a better score for competitive reasons.

While Phoronix.com and the development of the Phoronix Test Suite are handled by separate but linked entities, of interest to many Phoronix.com readers (and the other publications that leverage this software) will be the embeddable result graphs to be found on OpenBenchmarking.org and the improvements to be found within Iveland's pts_Graph library. There has long been a desire by some to have various statistical values and other technical metrics shown within the Phoronix Test Suite result graphs found on Phoronix.com and within the Phoronix Test Suite results viewer and other places. This will happen with OpenBenchmarking.org as these embeddable graphs will be rendered in SVG / JavaScript within supported browsers (and automatically fallback to PNG/HTML in other browsers) to allow for exposing in real-time individual result values, system logs, and other data. These far more advanced graphs will also be found within the Phoronix Test Suite results viewer on the client side, which is also receiving other new features to be talked about later on in the Iveland development cycle. Any other individuals and publications can also use these OpenBenchmarking.org embeddable graphs, which should lead to the most transparent testing processes within the industry when paired with the Phoronix Test Suite client and other components.

As expressed earlier, due to these very radical changes with Phoronix Test Suite 3.0 and the introduction of OpenBenchmarking.org, Iveland will not be introduced this quarter but rather will formally enter a production state in the first quarter of 2011. This is the first time in the Phoronix Test Suite history that a major update has not come out on a quarterly basis. Alpha and beta releases, however, should begin by late November and continue through early next year. OpenBenchmarking.org is just one of several major features to be found in Phoronix Test Suite 3.0 "Iveland", so stay tuned for more articles as we talk about the different features to further drive automated, open-source, collaborative testing on Linux, Solaris, BSD, Mac OS X, and Windows platforms.

About The Author
Michael Larabel is the principal author of Phoronix.com and founded the web-site in 2004 with a focus on enriching the Linux hardware experience and being the largest web-site devoted to Linux hardware reviews, particularly for products relevant to Linux gamers and enthusiasts but also commonly reviewing servers/workstations and embedded Linux devices. Michael has written more than 10,000 articles covering the state of Linux hardware support, Linux performance, graphics hardware drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org automated testing software. He can be followed via and or contacted via .
Latest Linux Hardware Reviews
  1. MSI X99S SLI PLUS On Linux
  2. NVIDIA GeForce GTX 970 Offers Great Linux Performance
  3. CompuLab Intense-PC2: An Excellent, Fanless, Mini PC Powered By Intel's i7 Haswell
  4. From The Atom 330 To Haswell ULT: Intel Linux Performance Benchmarks
Latest Linux Articles
  1. Open-Source Radeon 2D Performance Is Better With Ubuntu 14.10
  2. RunAbove: A POWER8 Compute Cloud With Offerings Up To 176 Threads
  3. 6-Way Ubuntu 14.10 Linux Desktop Benchmarks
  4. Ubuntu 14.10 XMir System Compositor Benchmarks
Latest Linux News
  1. Dead Island GOTY Now Available On Linux/SteamOS
  2. Ubuntu 14.04 In The Power8 Cloud From RunAbove
  3. KDE With Theoretical Client-Side Decorations, Windows 10 Influence
  4. Sandusky Lee: Great Cabinets For Storing All Your Computer Gear
  5. Fedora 21 Beta & Final Release Slip Further
  6. Mesa 10.3.2 Has A Couple Bug-Fixes
  7. RadeonSI/R600g HyperZ Support Gets Turned Back On
  8. openSUSE Factory & Tumbleweed Are Merging
  9. More Fedora Delays: Fedora 21 Beta Slips
  10. Mono Brings C# To The Unreal Engine 4
Latest Forum Discussions
  1. Looking for a Open-Source AMD experienced Linux mentor
  2. Users/Developers Threatening Fork Of Debian GNU/Linux
  3. Updated and Optimized Ubuntu Free Graphics Drivers
  4. HOPE: The Ease Of Python With The Speed Of C++
  5. Use Ubuntu MATE 14.10 Make it an official distro.
  6. Debian Is Back To Discussing Init Systems, Freedom of Choice
  7. AMD Radeon VDPAU Video Performance With Gallium3D
  8. Ubuntu 16.04 Might Be The Distribution's Last 32-Bit Release

Close Advertisement

Close Advertisement