1. Computers
  2. Display Drivers
  3. Graphics Cards
  4. Memory
  5. Motherboards
  6. Processors
  7. Software
  8. Storage
  9. Operating Systems

Facebook RSS Twitter Twitter Google Plus

Phoronix Test Suite

OpenBenchmarking Benchmarking Platform
Phoromatic Test Orchestration

Making Compiler, Disk Testing More Reproducible


Published on 13 March 2012 01:16 PM EDT
Written by Michael Larabel in Phoronix

There's some enhancements to share that make compiler and disk testing more reproducible under Linux, plus many other features.

Available today is Phoronix Test Suite 3.8-Bygland Milestone 3, which should be the last development release before this major update goes gold before month's end. (Depending upon commit activity this week, it may end up being released one week from today.) Several of the enhancements to this latest Phoronix Test Suite 3.8 version are not only relevant if you are a user of this open-source benchmarking software yourself, but even if you just look for Linux / BSD / Solaris / Windows benchmark results on Phoronix and the other publications that use the Phoronix Test Suite.

As I shared last week, there's been many compiler testing enhancements made within the Phoronix Test Suite framework during the 3.8-Bygland cycle for one of the commercial customers. Fortunately, this company is letting the work be committed to the open-source upstream version of the Phoronix Test Suite for all users to benefit. Since last week there's been more compiler testing enhancements, plus some disk testing enhancements, and there will be much more work committed in the coming weeks as work on Phoronix Test Suite 4.0 heats up, which they are continuing to support.

If you didn't read last week's article, read it, but the work then was for ensuring the C/Fortran/C++ compiler flags when any test is installed is easily available and rendered on the graphs showing the test results. Plus if any of the compiler flags changed between test results, there's special enhancements for highlighting the differences, etc. Here's a very example simply showing the important compiler flags being used by a given test:

As the latest work since then, the compiler's build configuration is also being automatically and seamlessly tracked by the Phoronix Test Suite. If there's any changes in the compiler's build configuration between test, it will be shown. Here's an example of that latest development:

That's the short, parsed form of the compiler's build configuration. Clicking upon that will then yield the full CC output, for example, when available (The system logs have long been captured by the Phoronix Test Suite, this contracted work is just about exposing more of it in an easy and attractive manner). [This is all for the SVG version of the tables and graphs. If you're browser doesn't yet fully support SVG, it will fall-back to the less-rich PNG graphs.] There's also some other ongoing compiler reporting work for highlighting the differences, etc. Feedback is also always welcome.

Committed yesterday is similar support to the compiler reporting improvements, but on the disk side. If a disk test is run, the kernel's scheduler is automatically reported plus the file-system mount options for wherever the test(s) were installed, as shown on the example above. As with most Phoronix Test Suite features, it's all handled automatically.

There's more system hardware / software reporting advancements being worked on, this is just the start of the work for this PTS Commercial client that's being made available to the public. A few more items may land prior to the Phoronix Test Suite 3.8-Bygland while the rest should be ready for Phoronix Test Suite 4.0. Again, the Phoronix Test Suite has long captured these system logs and other outputs, but most of this work will be centered around making it more presentable and easier to analyze when reproducing tests.

With the Phoronix Test Suite being focused upon fully automating tests, trying out these latest features on 3.8 Milestone 3 is as easy as running a command like phoronix-test-suite benchmark c-ray postmark. That command will download and install the C-Ray and PostMark tests followed by executing them in a fully automated manner under supported operating systems.

These reporting improvements aren't all that there is though to Phoronix Test Suite 3.8 Milestone 3, but there is much more. Some of the other new features include:

- The introduction of a look-aside cache and last-minute look-aside cache. Even prior to Phoronix Test Suite 1.0, there's been support for handling a download cache within the system, on a removable storage device, or via a networked interface. This allows those with many systems or routinely installing tests to house all of their needed test files locally, in order to eliminate needing to download files remotely and thus speeds up the install/setup process. (The HTML/PDF documentation covers how to setup a Phoronix Test Suite download cache.) The new look-aside cache support now looks at other installed test profiles to compare hashes of files in their download lists in an attempt to mirror matching files to eliminate downloading multiple copies or having to re-download files when installing a newer version of a test profile in a separate location (such as when testing multiple disk drives).

- Better detection of potential premature test failure and invalid results.

- Various bug-fixes.

- Refinements to the graphing improvements and the new SVG DOM IR implementation.

- Initial work towards supporting Minix 3.2.0. I've casually been working on seeing that the Phoronix Test Suite supports Minix along side its existing Linux, BSD, Hurd, Solaris, Windows, and Mac OS X operating system support. Within this third milestone release there's some basic hardware detection support, but more work is still left to be done before Minix would be fully supported by this GPLv3-licensed software.

- The graphics event checker module has been largely re-written. This module is loaded by the Phoronix Test Suite when a graphics test is being run and ensures that afterwards the system is still left in a clean state. In particular, it checks various kernel/driver interfaces for GPU errors and also ensures the system's resolution (if changed) was restored to its proper mode.

Plus there's various other changes and this is all on top of the Phoronix Test Suite 3.8 improvements made during milestone 1 and milestone 2 with items like improved graph generation and Mac OS X 10.8 support.

Feel free to give Phoronix Test Suite 3.8-Bygland a try at Phoronix-Test-Suite.com. Also be sure to share your results, tests, and suites via OpenBenchmarking.org.

For anyone wishing to learn more about the Phoronix Test Suite for personal or commercial purposes (or are just an active Phoronix readers / community members), this weekend through the middle of next week I'll be in San Diego and La Jolla for meet-ups. The Linux Foundation Collaboration Summit is also coming up again in early April. Contact me for that or PTS Commercial for other matters.

Look for more announcements soon with the official Phoronix Test Suite 3.8 release happening in a matter of days.

About The Author
Michael Larabel is the principal author of Phoronix.com and founded the web-site in 2004 with a focus on enriching the Linux hardware experience and being the largest web-site devoted to Linux hardware reviews, particularly for products relevant to Linux gamers and enthusiasts but also commonly reviewing servers/workstations and embedded Linux devices. Michael has written more than 10,000 articles covering the state of Linux hardware support, Linux performance, graphics hardware drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org automated testing software. He can be followed via and or contacted via .
Latest Linux News
  1. Pinos Is For Linux Video What PulseAudio Is For Audio
  2. Crossing 200,000 Benchmark Results Posted On LinuxBenchmarking.com
  3. New Mesa Vec4 Backend For Intel, Supports Their NIR Goals
  4. "PulseVideo" Coming To Complement PulseAudio?
  5. Premium Users Now Can Experience Our New Site
  6. XFS Will Get DAX Support In The Linux 4.2 Kernel
  7. X.Org Server Lands More Mode-Setting/GLAMOR Improvements, But No Sign Of 1.18
  8. Linux Mint 17.2 Officially Released With Cinnamon/MATE Flavors
  9. Fedora For MIPS Is Now Out In Testing, Supports The Creator CI20
  10. KDE Plasma 5.3.2 Fixes Shutdown Scripts, Few Dozen Other Bugs
Latest Articles & Reviews
  1. How KDE VDG Is Trying To Make Open-Source Software Beautiful
  2. Attempting To Try Out BCache On The Linux 4.1 Kernel
  3. CompuLab's Fitlet Is A Very Tiny, Fanless, Linux PC With AMD A10 Micro
  4. AMD A10-7870K Godavari: RadeonSI Gallium3D vs. Catalyst Linux Drivers
Most Viewed News This Week
  1. Kubuntu 15.10 Could Be The End Of The Road
  2. Linus Is Looking Forward To Merging KDBUS, But Not Convinced By Performance
  3. NVIDIA Starts Supplying Open-Source Hardware Reference Headers
  4. KDBUS Won't Be Pushed Until The Linux 4.3 Kernel
  5. Linux 4.2 Kernel Gets Port To New Processor Architecture
  6. The Staging Pull For Linux 4.2: "Big, Really Big"
  7. The State & Complications Of Porting The Unity Editor To Linux
  8. SteamOS "Brewmaster" Is Valve's New Debian 8.1 Based Version