1. Computers
  2. Display Drivers
  3. Graphics Cards
  4. Memory
  5. Motherboards
  6. Processors
  7. Software
  8. Storage
  9. Operating Systems


Facebook RSS Twitter Twitter Google Plus


Phoronix Test Suite

OpenBenchmarking.org

Making Compiler, Disk Testing More Reproducible

Phoronix

Published on 13 March 2012 01:16 PM EDT
Written by Michael Larabel in Phoronix
7 Comments

There's some enhancements to share that make compiler and disk testing more reproducible under Linux, plus many other features.

Available today is Phoronix Test Suite 3.8-Bygland Milestone 3, which should be the last development release before this major update goes gold before month's end. (Depending upon commit activity this week, it may end up being released one week from today.) Several of the enhancements to this latest Phoronix Test Suite 3.8 version are not only relevant if you are a user of this open-source benchmarking software yourself, but even if you just look for Linux / BSD / Solaris / Windows benchmark results on Phoronix and the other publications that use the Phoronix Test Suite.

As I shared last week, there's been many compiler testing enhancements made within the Phoronix Test Suite framework during the 3.8-Bygland cycle for one of the commercial customers. Fortunately, this company is letting the work be committed to the open-source upstream version of the Phoronix Test Suite for all users to benefit. Since last week there's been more compiler testing enhancements, plus some disk testing enhancements, and there will be much more work committed in the coming weeks as work on Phoronix Test Suite 4.0 heats up, which they are continuing to support.

If you didn't read last week's article, read it, but the work then was for ensuring the C/Fortran/C++ compiler flags when any test is installed is easily available and rendered on the graphs showing the test results. Plus if any of the compiler flags changed between test results, there's special enhancements for highlighting the differences, etc. Here's a very example simply showing the important compiler flags being used by a given test:

As the latest work since then, the compiler's build configuration is also being automatically and seamlessly tracked by the Phoronix Test Suite. If there's any changes in the compiler's build configuration between test, it will be shown. Here's an example of that latest development:

That's the short, parsed form of the compiler's build configuration. Clicking upon that will then yield the full CC output, for example, when available (The system logs have long been captured by the Phoronix Test Suite, this contracted work is just about exposing more of it in an easy and attractive manner). [This is all for the SVG version of the tables and graphs. If you're browser doesn't yet fully support SVG, it will fall-back to the less-rich PNG graphs.] There's also some other ongoing compiler reporting work for highlighting the differences, etc. Feedback is also always welcome.

Committed yesterday is similar support to the compiler reporting improvements, but on the disk side. If a disk test is run, the kernel's scheduler is automatically reported plus the file-system mount options for wherever the test(s) were installed, as shown on the example above. As with most Phoronix Test Suite features, it's all handled automatically.

There's more system hardware / software reporting advancements being worked on, this is just the start of the work for this PTS Commercial client that's being made available to the public. A few more items may land prior to the Phoronix Test Suite 3.8-Bygland while the rest should be ready for Phoronix Test Suite 4.0. Again, the Phoronix Test Suite has long captured these system logs and other outputs, but most of this work will be centered around making it more presentable and easier to analyze when reproducing tests.

With the Phoronix Test Suite being focused upon fully automating tests, trying out these latest features on 3.8 Milestone 3 is as easy as running a command like phoronix-test-suite benchmark c-ray postmark. That command will download and install the C-Ray and PostMark tests followed by executing them in a fully automated manner under supported operating systems.

These reporting improvements aren't all that there is though to Phoronix Test Suite 3.8 Milestone 3, but there is much more. Some of the other new features include:

- The introduction of a look-aside cache and last-minute look-aside cache. Even prior to Phoronix Test Suite 1.0, there's been support for handling a download cache within the system, on a removable storage device, or via a networked interface. This allows those with many systems or routinely installing tests to house all of their needed test files locally, in order to eliminate needing to download files remotely and thus speeds up the install/setup process. (The HTML/PDF documentation covers how to setup a Phoronix Test Suite download cache.) The new look-aside cache support now looks at other installed test profiles to compare hashes of files in their download lists in an attempt to mirror matching files to eliminate downloading multiple copies or having to re-download files when installing a newer version of a test profile in a separate location (such as when testing multiple disk drives).

- Better detection of potential premature test failure and invalid results.

- Various bug-fixes.

- Refinements to the graphing improvements and the new SVG DOM IR implementation.

- Initial work towards supporting Minix 3.2.0. I've casually been working on seeing that the Phoronix Test Suite supports Minix along side its existing Linux, BSD, Hurd, Solaris, Windows, and Mac OS X operating system support. Within this third milestone release there's some basic hardware detection support, but more work is still left to be done before Minix would be fully supported by this GPLv3-licensed software.

- The graphics event checker module has been largely re-written. This module is loaded by the Phoronix Test Suite when a graphics test is being run and ensures that afterwards the system is still left in a clean state. In particular, it checks various kernel/driver interfaces for GPU errors and also ensures the system's resolution (if changed) was restored to its proper mode.

Plus there's various other changes and this is all on top of the Phoronix Test Suite 3.8 improvements made during milestone 1 and milestone 2 with items like improved graph generation and Mac OS X 10.8 support.

Feel free to give Phoronix Test Suite 3.8-Bygland a try at Phoronix-Test-Suite.com. Also be sure to share your results, tests, and suites via OpenBenchmarking.org.

For anyone wishing to learn more about the Phoronix Test Suite for personal or commercial purposes (or are just an active Phoronix readers / community members), this weekend through the middle of next week I'll be in San Diego and La Jolla for meet-ups. The Linux Foundation Collaboration Summit is also coming up again in early April. Contact me for that or PTS Commercial for other matters.

Look for more announcements soon with the official Phoronix Test Suite 3.8 release happening in a matter of days.

About The Author
Michael Larabel is the principal author of Phoronix.com and founded the web-site in 2004 with a focus on enriching the Linux hardware experience and being the largest web-site devoted to Linux hardware reviews, particularly for products relevant to Linux gamers and enthusiasts but also commonly reviewing servers/workstations and embedded Linux devices. Michael has written more than 10,000 articles covering the state of Linux hardware support, Linux performance, graphics hardware drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org automated testing software. He can be followed via and or contacted via .
Latest Linux Hardware Reviews
  1. MSI X99S SLI PLUS On Linux
  2. NVIDIA GeForce GTX 970 Offers Great Linux Performance
  3. CompuLab Intense-PC2: An Excellent, Fanless, Mini PC Powered By Intel's i7 Haswell
  4. From The Atom 330 To Haswell ULT: Intel Linux Performance Benchmarks
Latest Linux Articles
  1. Open-Source Radeon 2D Performance Is Better With Ubuntu 14.10
  2. RunAbove: A POWER8 Compute Cloud With Offerings Up To 176 Threads
  3. 6-Way Ubuntu 14.10 Linux Desktop Benchmarks
  4. Ubuntu 14.10 XMir System Compositor Benchmarks
Latest Linux News
  1. Dead Island GOTY Now Available On Linux/SteamOS
  2. Ubuntu 14.04 In The Power8 Cloud From RunAbove
  3. KDE With Theoretical Client-Side Decorations, Windows 10 Influence
  4. Sandusky Lee: Great Cabinets For Storing All Your Computer Gear
  5. Fedora 21 Beta & Final Release Slip Further
  6. Mesa 10.3.2 Has A Couple Bug-Fixes
  7. RadeonSI/R600g HyperZ Support Gets Turned Back On
  8. openSUSE Factory & Tumbleweed Are Merging
  9. More Fedora Delays: Fedora 21 Beta Slips
  10. Mono Brings C# To The Unreal Engine 4
Latest Forum Discussions
  1. Updated and Optimized Ubuntu Free Graphics Drivers
  2. HOPE: The Ease Of Python With The Speed Of C++
  3. Use Ubuntu MATE 14.10 Make it an official distro.
  4. Users/Developers Threatening Fork Of Debian GNU/Linux
  5. Debian Is Back To Discussing Init Systems, Freedom of Choice
  6. AMD Radeon VDPAU Video Performance With Gallium3D
  7. Ubuntu 16.04 Might Be The Distribution's Last 32-Bit Release
  8. Linux hacker compares Solaris kernel code: