1. Computers
  2. Display Drivers
  3. Graphics Cards
  4. Memory
  5. Motherboards
  6. Processors
  7. Software
  8. Storage
  9. Operating Systems


Facebook RSS Twitter Twitter Google Plus


Phoronix Test Suite

OpenBenchmarking.org

Building A Benchmarking Test Farm With Phoromatic

Michael Larabel

Published on 4 November 2009
Written by Michael Larabel
Page 2 of 2 - 3 Comments

The Phoromatic web interface also supports comparing multiple test results all from your web browser, whether it is to compare the results from a same schedule across multiple systems, or to look at how the performance has changed of a test system over the course of time. To compare multiple test results, simply click on the tag icon in the front of each results row and then click on the header above that table (i.e. "Latest Test Results"). Features like being able to analyze batch runs, line graphs showing how the performance of test system(s) with a given test schedule have changed over the course of time, an option to issue a test call when a new revision control commit is made to a particular project (i.e. new kernel or X Server commit in Git) rather than on a timed basis (event-driven testing), and other statistical/analytical features will also be turned on within Phoromatic in the near future. For now we are ensuring all of the core functionality is solid and to see what features the Phoromatic users are most interested in utilizing. Various other features are also on the way such as the ability to clone results from a private Phoromatic repository, advanced search / sorting capabilities, and an option for anonymous uploads to a public Phoromatic repository. The enterprise version of Phoromatic will also allow the Phoromatic server to be installed within a corporate network, to support multiple user accounts for managing the web-interface, and features to better integrate Phoromatic within an already existing in-house testing environment.

From the web-interface, Phoromatic also shows all linked-up test systems along with their installed software/hardware, which is maintained in real-time and shown are any test errors/warnings generated on that test system along with the current tasks of that test system.

The core of Phoromatic is also going to be used to power the next-generation version of Phoronix Global, which will launch in 2010 with a plethora of new features and improvements, and picking up some of the Phoromatic features like web-based results comparison, AJAX-driven menus, sort-able tables, email notifications, better search capabilities, and other analytical capabilities.

Whether you are just an enthusiast using the Phoronix Test Suite on a couple of systems or are a company looking to automate the scheduled execution of Phoronix Test Suite tests across a multitude of systems, we invite you to try out this public beta of Phoromatic.com and to supply feedback to us as we have a platter of new features to still implement. With Phoromatic, it should be incredibly easy to assemble your own "benchmarking test farm" and to enjoy its robust set of features.

As another step in our global benchmarking crusade, a new Phoronix Test Suite product currently in brainstorming is codenamed "Phorotrack". Phorotrack is a plug-in for Phoromatic to provide additional features and options for tracking the performance of a particular software component over time. As part of Phorotrack, we would also be looking to offer public performance tracking figures for key software projects and distributions. This would be using our own in-house test farm at Phoronix.com both on real hardware and in a virtualized environment -- to effectively reach a point of "cloud benchmarking" for regression monitoring.

Projects using Phorotrack could then connect in and request particular test runs from the available Phoronix Test Suite profiles, which then would begin running routinely from our farm and report the results back to them, in order to aide them with monitoring of important and relevant performance metrics. Outside individuals could also connect in to donate computing power towards performance testing for a particular project, effectively like Folding@HOME but for community benchmarking of open-source software. In fact, already later this month as an experiment we may begin tracking the performance of a few key Linux distributions with their nightly builds on a daily basis through Phoromatic and this add-on.

Phoronix Test Suite 2.2 with Phoromatic support and many other new features (improved statistical significance, network proxy support, test recovery support, autonomous Git bisecting support of performance regressions, an improved GTK GUI, etc) will be officially released in November. PTS Desktop Live 2009.4 is also on the way, and of course there's even more exciting work coming down the pipe in 2010 -- Windows 7 benchmarking support could even come.

About The Author
Michael Larabel is the principal author of Phoronix.com and founded the web-site in 2004 with a focus on enriching the Linux hardware experience and being the largest web-site devoted to Linux hardware reviews, particularly for products relevant to Linux gamers and enthusiasts but also commonly reviewing servers/workstations and embedded Linux devices. Michael has written more than 10,000 articles covering the state of Linux hardware support, Linux performance, graphics hardware drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org automated testing software. He can be followed via and or contacted via .
Latest Linux Hardware Reviews
  1. AMD Radeon R9 290: Gallium3D vs. Catalyst Drivers
  2. AMD Radeon R9 290 Open-Source Driver Works, But Has A Ways To Go
  3. Trying The Configurable 45 Watt TDP With AMD's A10-7800 / A6-7400K
  4. Sumo's Omni Gets Reloaded
Latest Linux Articles
  1. Preview: OS X 10.10 Yosemite vs. Ubuntu Linux GPU Performance
  2. Radeon Graphics Yield Mixed Results With Linux 3.17 Kernel
  3. AMD's RadeonSI Driver Sped Up A Lot This Summer
  4. Intel's Latest Linux Graphics Code Competes Against OS X 10.9
Latest Linux News
  1. Google Chrome 37 Brings Many Security Fixes
  2. MenuetOS Updated With SMP Threads & Onscreen Keyboard
  3. Mesa Has A New Release Manager
  4. Enlightenment E19 Lands Its New Wayland Compositor Code
  5. Nouveau Turns Into A Mess In Latest Linux 3.17 + Mesa 10.3-dev Tests
  6. New GCC 5.0 Changes, Command-Line Options That Landed So Far
  7. SteamOS Update 133 Has Better Intel Performance, VA-API
  8. DRM Graphics Changes For Linux 3.18 Might End Up Being Smaller
  9. Ioquake3 Works On Finally Switching Over To SDL2
  10. Linux 3.17-rc2 Release Celebrates 23 Years Of Linux
Latest Forum Discussions
  1. Updated and Optimized Ubuntu Free Graphics Drivers
  2. AMD Releases UVD Video Decode Support For R600 GPUs
  3. Announcing radeontop, a tool for viewing the GPU usage
  4. Users defect to Linux as OpenBSD removes Lynx from base system
  5. Chinese People Try To Patent Wine On ARM
  6. American Citizens running AMOK for food stamps
  7. "The World's Most Highly-Assured OS" Kernel Open-Sourced
  8. What Linux Distribution Should Be Benchmarked The Most?