1. Computers
  2. Display Drivers
  3. Graphics Cards
  4. Memory
  5. Motherboards
  6. Processors
  7. Software
  8. Storage
  9. Operating Systems


Facebook RSS Twitter Twitter Google Plus


Phoronix Test Suite

OpenBenchmarking Benchmarking Platform
Phoromatic Test Orchestration

Building A Benchmarking Test Farm With Phoromatic

Michael Larabel

Published on 4 November 2009
Written by Michael Larabel
Page 2 of 2 - 3 Comments

The Phoromatic web interface also supports comparing multiple test results all from your web browser, whether it is to compare the results from a same schedule across multiple systems, or to look at how the performance has changed of a test system over the course of time. To compare multiple test results, simply click on the tag icon in the front of each results row and then click on the header above that table (i.e. "Latest Test Results"). Features like being able to analyze batch runs, line graphs showing how the performance of test system(s) with a given test schedule have changed over the course of time, an option to issue a test call when a new revision control commit is made to a particular project (i.e. new kernel or X Server commit in Git) rather than on a timed basis (event-driven testing), and other statistical/analytical features will also be turned on within Phoromatic in the near future. For now we are ensuring all of the core functionality is solid and to see what features the Phoromatic users are most interested in utilizing. Various other features are also on the way such as the ability to clone results from a private Phoromatic repository, advanced search / sorting capabilities, and an option for anonymous uploads to a public Phoromatic repository. The enterprise version of Phoromatic will also allow the Phoromatic server to be installed within a corporate network, to support multiple user accounts for managing the web-interface, and features to better integrate Phoromatic within an already existing in-house testing environment.

From the web-interface, Phoromatic also shows all linked-up test systems along with their installed software/hardware, which is maintained in real-time and shown are any test errors/warnings generated on that test system along with the current tasks of that test system.

The core of Phoromatic is also going to be used to power the next-generation version of Phoronix Global, which will launch in 2010 with a plethora of new features and improvements, and picking up some of the Phoromatic features like web-based results comparison, AJAX-driven menus, sort-able tables, email notifications, better search capabilities, and other analytical capabilities.

Whether you are just an enthusiast using the Phoronix Test Suite on a couple of systems or are a company looking to automate the scheduled execution of Phoronix Test Suite tests across a multitude of systems, we invite you to try out this public beta of Phoromatic.com and to supply feedback to us as we have a platter of new features to still implement. With Phoromatic, it should be incredibly easy to assemble your own "benchmarking test farm" and to enjoy its robust set of features.

As another step in our global benchmarking crusade, a new Phoronix Test Suite product currently in brainstorming is codenamed "Phorotrack". Phorotrack is a plug-in for Phoromatic to provide additional features and options for tracking the performance of a particular software component over time. As part of Phorotrack, we would also be looking to offer public performance tracking figures for key software projects and distributions. This would be using our own in-house test farm at Phoronix.com both on real hardware and in a virtualized environment -- to effectively reach a point of "cloud benchmarking" for regression monitoring.

Projects using Phorotrack could then connect in and request particular test runs from the available Phoronix Test Suite profiles, which then would begin running routinely from our farm and report the results back to them, in order to aide them with monitoring of important and relevant performance metrics. Outside individuals could also connect in to donate computing power towards performance testing for a particular project, effectively like Folding@HOME but for community benchmarking of open-source software. In fact, already later this month as an experiment we may begin tracking the performance of a few key Linux distributions with their nightly builds on a daily basis through Phoromatic and this add-on.

Phoronix Test Suite 2.2 with Phoromatic support and many other new features (improved statistical significance, network proxy support, test recovery support, autonomous Git bisecting support of performance regressions, an improved GTK GUI, etc) will be officially released in November. PTS Desktop Live 2009.4 is also on the way, and of course there's even more exciting work coming down the pipe in 2010 -- Windows 7 benchmarking support could even come.

About The Author
Michael Larabel is the principal author of Phoronix.com and founded the web-site in 2004 with a focus on enriching the Linux hardware experience and being the largest web-site devoted to Linux hardware reviews, particularly for products relevant to Linux gamers and enthusiasts but also commonly reviewing servers/workstations and embedded Linux devices. Michael has written more than 10,000 articles covering the state of Linux hardware support, Linux performance, graphics hardware drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org automated testing software. He can be followed via and or contacted via .
Latest Articles & Reviews
  1. GLAMOR + RadeonSI 2D Acceleration Is Quite Good For Open-Source AMD 2D Performance
  2. AMD Radeon R9 290 OpenGL On Ubuntu 15.04: Catalyst vs. RadeonSI Gallium3D
  3. Ubuntu 15.04 Offers Faster OpenGL For AMD Radeon GPUs On Open-Source
  4. Ubuntu 15.04 Brings Some Graphics Performance Improvements For Intel Haswell
  5. Sub-$20 802.11n USB WiFi Adapter That's Linux Friendly
  6. The Lenovo T450s Is Working Beautifully With Linux
Latest Linux News
  1. Linux 4.1-RC2 Kernel Released
  2. GNOME 3.17.1 Released
  3. A Lot Of Improvements Are Coming For Mir 0.13, Including Work Towards Libinput
  4. Mobile Optimizations Coming For Phoronix
  5. Wayland 1.8 Alpha Release Delayed
  6. Godot Game Engine 1.1 Up To RC State
  7. ATI Rage128 Driver Now Has RandR Support
  8. Microsoft's Visual C++ Team Is Improving Clang For Windows
  9. Kodi 15.0 Beta 1 Released
  10. Lucid Sleep Support Is Being Worked On For The Upstream Linux Kernel
Most Viewed News This Week
  1. Trying Out Microsoft Visual Studio Code On Linux
  2. Microsoft Releases New Code IDE For Linux!
  3. Improvements On The Way For GNOME's Nautilus File Manager
  4. Kodi 15.0 Beta 1 Released
  5. GCC 4.9.2 vs. GCC 5 Benchmarks On An Intel Xeon Haswell
  6. QEMU 2.3 Officially Released
  7. Mono 4.0 Makes Use Of Microsoft's Open-Source Code, C# 6.0
  8. Debian GNU/Hurd 2015 Released