1. Computers
  2. Display Drivers
  3. Graphics Cards
  4. Memory
  5. Motherboards
  6. Processors
  7. Software
  8. Storage
  9. Operating Systems


Facebook RSS Twitter Twitter Google Plus


Phoronix Test Suite

OpenBenchmarking.org

Building A Benchmarking Test Farm With Phoromatic

Michael Larabel

Published on 4 November 2009
Written by Michael Larabel
Page 2 of 2 - 3 Comments

The Phoromatic web interface also supports comparing multiple test results all from your web browser, whether it is to compare the results from a same schedule across multiple systems, or to look at how the performance has changed of a test system over the course of time. To compare multiple test results, simply click on the tag icon in the front of each results row and then click on the header above that table (i.e. "Latest Test Results"). Features like being able to analyze batch runs, line graphs showing how the performance of test system(s) with a given test schedule have changed over the course of time, an option to issue a test call when a new revision control commit is made to a particular project (i.e. new kernel or X Server commit in Git) rather than on a timed basis (event-driven testing), and other statistical/analytical features will also be turned on within Phoromatic in the near future. For now we are ensuring all of the core functionality is solid and to see what features the Phoromatic users are most interested in utilizing. Various other features are also on the way such as the ability to clone results from a private Phoromatic repository, advanced search / sorting capabilities, and an option for anonymous uploads to a public Phoromatic repository. The enterprise version of Phoromatic will also allow the Phoromatic server to be installed within a corporate network, to support multiple user accounts for managing the web-interface, and features to better integrate Phoromatic within an already existing in-house testing environment.

From the web-interface, Phoromatic also shows all linked-up test systems along with their installed software/hardware, which is maintained in real-time and shown are any test errors/warnings generated on that test system along with the current tasks of that test system.

The core of Phoromatic is also going to be used to power the next-generation version of Phoronix Global, which will launch in 2010 with a plethora of new features and improvements, and picking up some of the Phoromatic features like web-based results comparison, AJAX-driven menus, sort-able tables, email notifications, better search capabilities, and other analytical capabilities.

Whether you are just an enthusiast using the Phoronix Test Suite on a couple of systems or are a company looking to automate the scheduled execution of Phoronix Test Suite tests across a multitude of systems, we invite you to try out this public beta of Phoromatic.com and to supply feedback to us as we have a platter of new features to still implement. With Phoromatic, it should be incredibly easy to assemble your own "benchmarking test farm" and to enjoy its robust set of features.

As another step in our global benchmarking crusade, a new Phoronix Test Suite product currently in brainstorming is codenamed "Phorotrack". Phorotrack is a plug-in for Phoromatic to provide additional features and options for tracking the performance of a particular software component over time. As part of Phorotrack, we would also be looking to offer public performance tracking figures for key software projects and distributions. This would be using our own in-house test farm at Phoronix.com both on real hardware and in a virtualized environment -- to effectively reach a point of "cloud benchmarking" for regression monitoring.

Projects using Phorotrack could then connect in and request particular test runs from the available Phoronix Test Suite profiles, which then would begin running routinely from our farm and report the results back to them, in order to aide them with monitoring of important and relevant performance metrics. Outside individuals could also connect in to donate computing power towards performance testing for a particular project, effectively like Folding@HOME but for community benchmarking of open-source software. In fact, already later this month as an experiment we may begin tracking the performance of a few key Linux distributions with their nightly builds on a daily basis through Phoromatic and this add-on.

Phoronix Test Suite 2.2 with Phoromatic support and many other new features (improved statistical significance, network proxy support, test recovery support, autonomous Git bisecting support of performance regressions, an improved GTK GUI, etc) will be officially released in November. PTS Desktop Live 2009.4 is also on the way, and of course there's even more exciting work coming down the pipe in 2010 -- Windows 7 benchmarking support could even come.

About The Author
Michael Larabel is the principal author of Phoronix.com and founded the web-site in 2004 with a focus on enriching the Linux hardware experience and being the largest web-site devoted to Linux hardware reviews, particularly for products relevant to Linux gamers and enthusiasts but also commonly reviewing servers/workstations and embedded Linux devices. Michael has written more than 10,000 articles covering the state of Linux hardware support, Linux performance, graphics hardware drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org automated testing software. He can be followed via and or contacted via .
Latest Linux Hardware Reviews
  1. Intel Launches The Core i7 5960X, Mighty Powerful Haswell-E CPUs
  2. AMD Radeon R9 290: Gallium3D vs. Catalyst Drivers
  3. AMD Radeon R9 290 Open-Source Driver Works, But Has A Ways To Go
  4. Trying The Configurable 45 Watt TDP With AMD's A10-7800 / A6-7400K
Latest Linux Articles
  1. How Intel Graphics On Linux Compare To Open-Source AMD/NVIDIA Drivers
  2. The Fastest NVIDIA GPUs For Open-Source Nouveau With Steam Linux Gaming
  3. Testing For The Latest Linux Kernel Power Regression
  4. The Most Energy Efficient Radeon GPU For AMD Linux Gaming
Latest Linux News
  1. Nouveau X.Org Driver Released With DRI3+Present, Maxwell, GLAMOR
  2. Microsoft & AMD Release C++ AMP Compiler With Linux Support
  3. AMD, Wine & Valve Dominated August For Linux Users
  4. Linux 3.17-rc3 Kernel Released Back On Schedule
  5. Lennart Poettering Talks Up His New Linux Vision That Involves Btrfs
  6. Mesa 10.3 RC2 Arrives Via Its New Release Manager
  7. Ubuntu 14.10's Lack Of X.Org Server 1.16 Gets Blamed On AMD
  8. MSI Motherboard BIOS Updating Remains A Pain For Linux Users
  9. See How Your Linux System Performs Against The Latest Intel/AMD CPUs
  10. AMD Steppe Eagle Flys To Coreboot
Latest Forum Discussions
  1. SSD seems slow
  2. AMD Releases UVD Video Decode Support For R600 GPUs
  3. Is laptop with Intel CPU and AMD dGPU worth buying considering especially AMD Enduro?
  4. Radeon HD5670 and Ubuntu 14.04
  5. Btrfs Gets Talked Up, Googler Encourages You To Try Btrfs
  6. Updated graphics drivers for Ubuntu 12.04 Precise LTS
  7. Catalyst 14.201.1008
  8. It's Now Possible To Play Netflix Natively On Linux Without Wine Plug-Ins