1. Computers
  2. Display Drivers
  3. Graphics Cards
  4. Memory
  5. Motherboards
  6. Processors
  7. Software
  8. Storage
  9. Operating Systems


Facebook RSS Twitter Twitter Google Plus


Phoronix Test Suite

OpenBenchmarking.org

Going Beyond Just Measuring Frame Rates

Michael Larabel

Published on 17 November 2009
Written by Michael Larabel
Page 1 of 2 - 3 Comments

Yesterday marked the release of Phoronix Test Suite 2.2 and it was the best version yet with the addition of many new exciting and useful features. While this release was gratifying, there are much greater plans for the Phoronix Test Suite going into the next decade. It has already been shared that Windows support is coming, but there are other huge features coming too as soon as Q1'2010. Up to this point, most of the tests and the design of pts-core (the Phoronix Test Suite engine) have been focused on quantitative benchmarks with many of the tests spitting out a frame-rate, time, or some other measurement. However, now being supported in the Phoronix Test Suite is the ability to produce abstract results, such as screenshots used for image quality comparisons. The Phoronix Test Suite can now track the image quality of various test profiles (such as OpenGL games) across hardware configurations, drivers, and more. All of this is still leveraged upon the existing Phoronix Test Suite framework and our design philosophies so that even image quality comparisons can be carried out autonomously, the ability to compare many results side-by-side, support for carrying out these tests remotely via Phoromatic, and the ability to share your abstract results with others via Phoronix Global. Now not only can you be sure you are satisfied with the quantitative frame-rate of the hardware you have -- or are about to purchase -- but you have a plethora of options for looking at the qualitative performance too.

Right now most of the "abstract results" work that is already completed for Phoronix Test Suite 2.4 is for image quality comparisons, but other result types will continue to be added. For end-users, this allows them to not only use our software to find out the speed of a graphics card via an assortment of games and other 2D/3D tests found in the Phoronix Test Suite (there's 41 graphics test profiles right now!), but to also find out how the image quality compares between drivers and graphics cards. We intend to begin providing these image quality results within graphics card reviews and driver comparisons at Phoronix.com and hope the other publications using the Phoronix Test Suite will do the same as well. In an open letter to tech review sites we called out on fellow web-sites to begin testing hardware under Linux to complement their Windows testing considering the ease-of-use of the Phoronix Test Suite or how even easy it can get when using our Linux distribution. With image quality comparisons this remains just as easy to carry out. Image quality comparisons can also be done through Phoronix Global for an even larger comparison base from users around the world (though automated image uploads for Phoronix Global is currently disabled due to bandwidth concerns).

To developers and quality assurance engineers, this new support is even of greater benefit. Tests that produce image quality comparisons are carried out with the Phoronix Test Suite in the very same way that all other test profiles are processed. What this means is that any existing Phoronix Test Suite commands, PTS modules (thermal monitoring, etc), and other Phoronix Test Suite components will all work with these new test profiles. Image quality tests can be scheduled remotely through our remote test management software and all of that good stuff. What this also means is that these test profiles can also leverage the immense power offered to traditional test profiles producing quantitative results.

Below is an example rendering that was produced by the Phoronix Test Suite -- with our latest Phoronix Test Suite 2.4 "Lenvik" code as of today -- in its usual autonomous, clean, reproducible, and easy-to-use manner. This is an example from the iqc-nexuiz test profile, which runs Image Quality Comparison (IQC) tests atop the popular Nexuiz game. When running this test profile the user is prompted with what scene(s) to capture and the Phoronix Test Suite is off doing its magic. This test profile is one of the simplest yet most advanced test profiles found in the Phoronix Test Suite through its use of features like Cascading Test Profiles, cache shares, and other pts-core technologies.

This image (which was a simple save-as from the PTS Results Viewer and then compressed in The GIMP to conserve bandwidth) was produced by simply running phoronix-test-suite benchmark iqc-nexuiz and then running phoronix-test-suite benchmark iqc-nexuiz after installing a knowingly faulty driver configuration, as an example to show off the difference between two images. Easy as that. Those writing test profiles also have a set of options to now exploit with the Phoronix Test Suite for capturing screenshots with per-frame accuracy and storage without any compression, the automatic cropping of images based upon set data, varying image sizes, and other options. Those constructing test suites can also house quantitative and qualitative tests all within the same suite -- there is no limits being placed on qualitative tests.

<< Previous Page
1
Latest Linux Hardware Reviews
  1. Rosewill RS-MI-01: An Ultra Low-Cost Mini-ITX Chassis
  2. D-Link DCS-2330L HD Wireless Network Camera
  3. Gigabyte AM1M-S2H
  4. AMD's New Athlon/Semprons Give Old Phenom CPUs A Big Run For The Money
Latest Linux Articles
  1. AMD Catalyst 14.4 On Linux With Radeon R3 APU Graphics
  2. Red Hat Enterprise Linux 7.0 RC Benchmarks
  3. AMD Catalyst 14.4 Brings Few Linux Performance Improvements
  4. The Performance Of Fedora 20 Updated
Latest Linux News
  1. Intel Developer Proposes A New Linux CPU Load Metric
  2. R600 Gallium3D Lands Many OpenGL Fixes
  3. LLVMpipe Gallium3D Now Exposes GLSL 3.30
  4. NGINX 1.6 Brings SPDY 3.1 & Other New Features
  5. Linux Foundation Announces A Core Infrastructure Initiative
  6. More Steam Linux Tests/Benchmarks Might Be Coming
  7. NVIDIA's Amazing Single-Board ARM Computer Might Be Delayed
  8. Fedora 21 To Get A Playground, New Features
  9. PC-BSD Is Developing Its Own Desktop Environment
  10. Valve Is Bringing VOGL To Windows & Working On Regression Tests
  11. Canonical Is Taking Over Linux 3.13 Kernel Maintenance
  12. Google Web Designer Is Now Natively Available On Linux
Latest Forum Discussions
  1. Linux Kernel Developers Fed Up With Ridiculous Bugs In Systemd
  2. HTPC-upgrade advice: AMD Richland A8-7600 or Kaveri A10-6700T ???
  3. What Else Would You Like To See On Phoronix This Spring?
  4. The GNOME Foundation Is Running Short On Money
  5. The Most Amazing OpenGL Tech Demo In 64kb
  6. Announcing radeontop, a tool for viewing the GPU usage
  7. New card. Open source drivers only.
  8. Script for Fan Speed Control