1. Computers
  2. Display Drivers
  3. Graphics Cards
  4. Memory
  5. Motherboards
  6. Processors
  7. Software
  8. Storage
  9. Operating Systems


Facebook RSS Twitter Twitter Google Plus


Phoronix Test Suite

OpenBenchmarking.org

Making Compiler, Disk Testing More Reproducible

Phoronix

Published on 13 March 2012 01:16 PM EDT
Written by Michael Larabel in Phoronix
7 Comments

There's some enhancements to share that make compiler and disk testing more reproducible under Linux, plus many other features.

Available today is Phoronix Test Suite 3.8-Bygland Milestone 3, which should be the last development release before this major update goes gold before month's end. (Depending upon commit activity this week, it may end up being released one week from today.) Several of the enhancements to this latest Phoronix Test Suite 3.8 version are not only relevant if you are a user of this open-source benchmarking software yourself, but even if you just look for Linux / BSD / Solaris / Windows benchmark results on Phoronix and the other publications that use the Phoronix Test Suite.

As I shared last week, there's been many compiler testing enhancements made within the Phoronix Test Suite framework during the 3.8-Bygland cycle for one of the commercial customers. Fortunately, this company is letting the work be committed to the open-source upstream version of the Phoronix Test Suite for all users to benefit. Since last week there's been more compiler testing enhancements, plus some disk testing enhancements, and there will be much more work committed in the coming weeks as work on Phoronix Test Suite 4.0 heats up, which they are continuing to support.

If you didn't read last week's article, read it, but the work then was for ensuring the C/Fortran/C++ compiler flags when any test is installed is easily available and rendered on the graphs showing the test results. Plus if any of the compiler flags changed between test results, there's special enhancements for highlighting the differences, etc. Here's a very example simply showing the important compiler flags being used by a given test:

As the latest work since then, the compiler's build configuration is also being automatically and seamlessly tracked by the Phoronix Test Suite. If there's any changes in the compiler's build configuration between test, it will be shown. Here's an example of that latest development:

That's the short, parsed form of the compiler's build configuration. Clicking upon that will then yield the full CC output, for example, when available (The system logs have long been captured by the Phoronix Test Suite, this contracted work is just about exposing more of it in an easy and attractive manner). [This is all for the SVG version of the tables and graphs. If you're browser doesn't yet fully support SVG, it will fall-back to the less-rich PNG graphs.] There's also some other ongoing compiler reporting work for highlighting the differences, etc. Feedback is also always welcome.

Committed yesterday is similar support to the compiler reporting improvements, but on the disk side. If a disk test is run, the kernel's scheduler is automatically reported plus the file-system mount options for wherever the test(s) were installed, as shown on the example above. As with most Phoronix Test Suite features, it's all handled automatically.

There's more system hardware / software reporting advancements being worked on, this is just the start of the work for this PTS Commercial client that's being made available to the public. A few more items may land prior to the Phoronix Test Suite 3.8-Bygland while the rest should be ready for Phoronix Test Suite 4.0. Again, the Phoronix Test Suite has long captured these system logs and other outputs, but most of this work will be centered around making it more presentable and easier to analyze when reproducing tests.

With the Phoronix Test Suite being focused upon fully automating tests, trying out these latest features on 3.8 Milestone 3 is as easy as running a command like phoronix-test-suite benchmark c-ray postmark. That command will download and install the C-Ray and PostMark tests followed by executing them in a fully automated manner under supported operating systems.

These reporting improvements aren't all that there is though to Phoronix Test Suite 3.8 Milestone 3, but there is much more. Some of the other new features include:

- The introduction of a look-aside cache and last-minute look-aside cache. Even prior to Phoronix Test Suite 1.0, there's been support for handling a download cache within the system, on a removable storage device, or via a networked interface. This allows those with many systems or routinely installing tests to house all of their needed test files locally, in order to eliminate needing to download files remotely and thus speeds up the install/setup process. (The HTML/PDF documentation covers how to setup a Phoronix Test Suite download cache.) The new look-aside cache support now looks at other installed test profiles to compare hashes of files in their download lists in an attempt to mirror matching files to eliminate downloading multiple copies or having to re-download files when installing a newer version of a test profile in a separate location (such as when testing multiple disk drives).

- Better detection of potential premature test failure and invalid results.

- Various bug-fixes.

- Refinements to the graphing improvements and the new SVG DOM IR implementation.

- Initial work towards supporting Minix 3.2.0. I've casually been working on seeing that the Phoronix Test Suite supports Minix along side its existing Linux, BSD, Hurd, Solaris, Windows, and Mac OS X operating system support. Within this third milestone release there's some basic hardware detection support, but more work is still left to be done before Minix would be fully supported by this GPLv3-licensed software.

- The graphics event checker module has been largely re-written. This module is loaded by the Phoronix Test Suite when a graphics test is being run and ensures that afterwards the system is still left in a clean state. In particular, it checks various kernel/driver interfaces for GPU errors and also ensures the system's resolution (if changed) was restored to its proper mode.

Plus there's various other changes and this is all on top of the Phoronix Test Suite 3.8 improvements made during milestone 1 and milestone 2 with items like improved graph generation and Mac OS X 10.8 support.

Feel free to give Phoronix Test Suite 3.8-Bygland a try at Phoronix-Test-Suite.com. Also be sure to share your results, tests, and suites via OpenBenchmarking.org.

For anyone wishing to learn more about the Phoronix Test Suite for personal or commercial purposes (or are just an active Phoronix readers / community members), this weekend through the middle of next week I'll be in San Diego and La Jolla for meet-ups. The Linux Foundation Collaboration Summit is also coming up again in early April. Contact me for that or PTS Commercial for other matters.

Look for more announcements soon with the official Phoronix Test Suite 3.8 release happening in a matter of days.

About The Author
Michael Larabel is the principal author of Phoronix.com and founded the web-site in 2004 with a focus on enriching the Linux hardware experience and being the largest web-site devoted to Linux hardware reviews, particularly for products relevant to Linux gamers and enthusiasts but also commonly reviewing servers/workstations and embedded Linux devices. Michael has written more than 10,000 articles covering the state of Linux hardware support, Linux performance, graphics hardware drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org automated testing software. He can be followed via and or contacted via .
Latest Linux Hardware Reviews
  1. A Walkthrough Of The New 32 System Open-Source Linux Benchmarking Test Farm
  2. Habey MITX-6771: Mini-ITX Board With Quad-Core J1900 Bay Trail
  3. OCZ Vector 150 SSD On Linux
  4. Noctua i4 CPU Cooler: Great For Cooling High-End LGA-2011v3 CPUs
Latest Linux Articles
  1. AMD Kaveri: Open-Source Radeon Gallium3D vs. Catalyst 14.12 Omega Driver
  2. 12-Way AMD Catalyst 14.12 vs. NVIDIA 346 Series Linux GPU Comparison
  3. AMD Catalyst 14.12 Omega Driver Brings Mixed Results For Linux Users
  4. 6-Way Winter 2014 Linux Distribution Comparison
Latest Linux News
  1. Intel Skylake Audio Support For Linux 3.19
  2. After 10+ Years, NetworkManager Reaches v1.0
  3. VDPAU Updated To v0.9
  4. An Open Hardware Random Number Generator Proposed
  5. LLVM 3.6 Will Be Branched Next Month
  6. Opera Browser Puts Out Linux Updates For The Holidays
  7. GNOME Shell 3.15.3 Adds Support For High-Contrast Themes
  8. Linux 3.19: ThinkPad Muting Redone, New Dell Backlight Support, Acer Is Banging
  9. KVM Drops Support For IA64 While Adding Various x86 Improvements
  10. GCC 4.8.4 Officially Released
Latest Forum Discussions
  1. XLennart: A Game For Systemd Haters With Nothing Better To Do
  2. Need some hand holding with upgrading xserver
  3. Debian init discussion in Phoenix Wright format
  4. The New SuperTuxKart Looks Better, But Can Cause GPU/Driver Problems
  5. FPS capped on Linux (AMD fglrx drivers)
  6. Are there an app using HSA ?
  7. Bench specific mount point
  8. Tool for measuring FPS in games