The Phoronix Kernel Test Farm Is Alive

This year with the Phoronix Test Suite we have delivered four major updates to this leading, widely adopted, multi-platform testing software that has brought dozens of new test profiles and literally hundreds of significant changes. These changes ranged from features to autonomously track performance regressions within any code-base, the ability to not only compare frame-rates within OpenGL tests but image quality comparisons too, support for mobile platforms, and so much other major work to further drive automated testing and benchmarking not only on Linux but OpenSolaris, *BSD, and Mac OS X too. In 2009 we also launched PTS Desktop Live, our own operating system for carrying out standardized benchmarks in an easy-to-use and repeatable manner from a live Linux environment, and also Phoromatic, which is designed for the enterprise world and allows the Phoronix Test Suite to be easily deployed across many systems and then managed from a central interface. The year is not over yet, nor is our work on ensuring that the Phoronix Test Suite is the most powerful and robust testing/benchmarking platform. With that said, as of this morning our Phoronix kernel test farm is now alive!

As was mentioned in Building A Benchmarking Test Farm With Phoromatic, there are some grand plans in front of us and part of this includes better helping the Linux development community in monitoring the performance of their code and providing an incredible feature set on the Phoronix Test Suite platform for any developer or company to utilize. I wrote in that article last month:

"As another step in our global benchmarking crusade, a new Phoronix Test Suite product currently in brainstorming is codenamed 'Phorotrack'. Phorotrack is a plug-in for Phoromatic to provide additional features and options for tracking the performance of a particular software component over time. As part of Phorotrack, we would also be looking to offer public performance tracking figures for key software projects and distributions. This would be using our own in-house test farm at Phoronix.com both on real hardware and in a virtualized environment -- to effectively reach a point of "cloud benchmarking" for regression monitoring.

Projects using Phorotrack could then connect in and request particular test runs from the available Phoronix Test Suite profiles, which then would begin running routinely from our farm and report the results back to them, in order to aide them with monitoring of important and relevant performance metrics. Outside individuals could also connect in to donate computing power towards performance testing for a particular project, effectively like Folding@HOME but for community benchmarking of open-source software. In fact, already later this month as an experiment we may begin tracking the performance of a few key Linux distributions with their nightly builds on a daily basis through Phoromatic and this add-on."

The first step of that new endeavor is the Phoronix kernel test farm. As of this morning, the first system -- of hopefully many to come -- is online and operational. At least one more system should also go online this week in time for the Linux 2.6.33 merge window. These test systems are to automatically install the latest Linux kernel every morning (using the code from the Linus Git tree) and then they spend the day running benchmarks. With the set of tests currently being run, the testing process for each kernel every day runs for 13 to 14 hours! There are more than 50 tests being run through the Phoronix Test Suite and they include multiple disk, ray-tracing, computational biology, server, compression, audio/video encoding, physics, cryptography, and tests representing many other areas of interest. Results are then uploaded to Phoromatic automatically from this Linux test farm (temporarily being built out of my living room, actually).

These results will then be publicly available from this new Phoromatic extension that will be launching later this month or early next month (or earlier, if Phoromatic spots some notable regressions during the 2.6.33 merge window that are worth pointing out sooner). This Phoromatic extension will allow users to view all of the test results over a specified period, view system logs from a given day, etc. There is also some other exciting features to be announced. We believe this can and will become a vital resource to kernel developers and other areas as we branch out our performance analytic services. There are already Fortune 500 companies using Phoromatic, but many plans are still ahead. By the end of this month, there will hopefully be at least two more systems online that will be upgrading against Fedora Rawhide and Ubuntu Lucid every morning and carrying out performance tests in a similar manner to our Linux kernel benchmarking. Again, this testing farm is requiring zero user intervention and it is fully automated through the Phoronix Test Suite stack.

This is the latest service now being offered by Phoronix Media. If you appreciate this work being done, we ask you subscribe to Phoronix Premium, make a PayPal donation, use our shopping affiliate links, follow us on Twitter / Facebook, or even just drop by the forums and share your thoughts. Enterprise and consulting services are also offered for the Phoronix Test Suite.