1. Computers
  2. Display Drivers
  3. Graphics Cards
  4. Memory
  5. Motherboards
  6. Processors
  7. Software
  8. Storage
  9. Operating Systems


Facebook RSS Twitter Twitter Google Plus


Phoronix Test Suite

OpenBenchmarking.org

Building A Benchmarking Test Farm With Phoromatic

Michael Larabel

Published on 4 November 2009
Written by Michael Larabel
Page 2 of 2 - 3 Comments

The Phoromatic web interface also supports comparing multiple test results all from your web browser, whether it is to compare the results from a same schedule across multiple systems, or to look at how the performance has changed of a test system over the course of time. To compare multiple test results, simply click on the tag icon in the front of each results row and then click on the header above that table (i.e. "Latest Test Results"). Features like being able to analyze batch runs, line graphs showing how the performance of test system(s) with a given test schedule have changed over the course of time, an option to issue a test call when a new revision control commit is made to a particular project (i.e. new kernel or X Server commit in Git) rather than on a timed basis (event-driven testing), and other statistical/analytical features will also be turned on within Phoromatic in the near future. For now we are ensuring all of the core functionality is solid and to see what features the Phoromatic users are most interested in utilizing. Various other features are also on the way such as the ability to clone results from a private Phoromatic repository, advanced search / sorting capabilities, and an option for anonymous uploads to a public Phoromatic repository. The enterprise version of Phoromatic will also allow the Phoromatic server to be installed within a corporate network, to support multiple user accounts for managing the web-interface, and features to better integrate Phoromatic within an already existing in-house testing environment.

From the web-interface, Phoromatic also shows all linked-up test systems along with their installed software/hardware, which is maintained in real-time and shown are any test errors/warnings generated on that test system along with the current tasks of that test system.

The core of Phoromatic is also going to be used to power the next-generation version of Phoronix Global, which will launch in 2010 with a plethora of new features and improvements, and picking up some of the Phoromatic features like web-based results comparison, AJAX-driven menus, sort-able tables, email notifications, better search capabilities, and other analytical capabilities.

Whether you are just an enthusiast using the Phoronix Test Suite on a couple of systems or are a company looking to automate the scheduled execution of Phoronix Test Suite tests across a multitude of systems, we invite you to try out this public beta of Phoromatic.com and to supply feedback to us as we have a platter of new features to still implement. With Phoromatic, it should be incredibly easy to assemble your own "benchmarking test farm" and to enjoy its robust set of features.

As another step in our global benchmarking crusade, a new Phoronix Test Suite product currently in brainstorming is codenamed "Phorotrack". Phorotrack is a plug-in for Phoromatic to provide additional features and options for tracking the performance of a particular software component over time. As part of Phorotrack, we would also be looking to offer public performance tracking figures for key software projects and distributions. This would be using our own in-house test farm at Phoronix.com both on real hardware and in a virtualized environment -- to effectively reach a point of "cloud benchmarking" for regression monitoring.

Projects using Phorotrack could then connect in and request particular test runs from the available Phoronix Test Suite profiles, which then would begin running routinely from our farm and report the results back to them, in order to aide them with monitoring of important and relevant performance metrics. Outside individuals could also connect in to donate computing power towards performance testing for a particular project, effectively like Folding@HOME but for community benchmarking of open-source software. In fact, already later this month as an experiment we may begin tracking the performance of a few key Linux distributions with their nightly builds on a daily basis through Phoromatic and this add-on.

Phoronix Test Suite 2.2 with Phoromatic support and many other new features (improved statistical significance, network proxy support, test recovery support, autonomous Git bisecting support of performance regressions, an improved GTK GUI, etc) will be officially released in November. PTS Desktop Live 2009.4 is also on the way, and of course there's even more exciting work coming down the pipe in 2010 -- Windows 7 benchmarking support could even come.

About The Author
Michael Larabel is the principal author of Phoronix.com and founded the web-site in 2004 with a focus on enriching the Linux hardware experience and being the largest web-site devoted to Linux hardware reviews, particularly for products relevant to Linux gamers and enthusiasts but also commonly reviewing servers/workstations and embedded Linux devices. Michael has written more than 10,000 articles covering the state of Linux hardware support, Linux performance, graphics hardware drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org automated testing software. He can be followed via and or contacted via .
Latest Linux Hardware Reviews
  1. Rosewill RS-MI-01: An Ultra Low-Cost Mini-ITX Chassis
  2. D-Link DCS-2330L HD Wireless Network Camera
  3. Gigabyte AM1M-S2H
  4. AMD's New Athlon/Semprons Give Old Phenom CPUs A Big Run For The Money
Latest Linux Articles
  1. AMD Catalyst 14.4 On Linux With Radeon R3 APU Graphics
  2. Red Hat Enterprise Linux 7.0 RC Benchmarks
  3. AMD Catalyst 14.4 Brings Few Linux Performance Improvements
  4. The Performance Of Fedora 20 Updated
Latest Linux News
  1. Intel Developer Proposes A New Linux CPU Load Metric
  2. R600 Gallium3D Lands Many OpenGL Fixes
  3. LLVMpipe Gallium3D Now Exposes GLSL 3.30
  4. NGINX 1.6 Brings SPDY 3.1 & Other New Features
  5. Linux Foundation Announces A Core Infrastructure Initiative
  6. More Steam Linux Tests/Benchmarks Might Be Coming
  7. NVIDIA's Amazing Single-Board ARM Computer Might Be Delayed
  8. Fedora 21 To Get A Playground, New Features
  9. PC-BSD Is Developing Its Own Desktop Environment
  10. Valve Is Bringing VOGL To Windows & Working On Regression Tests
  11. Canonical Is Taking Over Linux 3.13 Kernel Maintenance
  12. Google Web Designer Is Now Natively Available On Linux
Latest Forum Discussions
  1. Linux Kernel Developers Fed Up With Ridiculous Bugs In Systemd
  2. HTPC-upgrade advice: AMD Richland A8-7600 or Kaveri A10-6700T ???
  3. What Else Would You Like To See On Phoronix This Spring?
  4. The GNOME Foundation Is Running Short On Money
  5. The Most Amazing OpenGL Tech Demo In 64kb
  6. Announcing radeontop, a tool for viewing the GPU usage
  7. New card. Open source drivers only.
  8. Script for Fan Speed Control