1. Computers
  2. Display Drivers
  3. Graphics Cards
  4. Memory
  5. Motherboards
  6. Processors
  7. Software
  8. Storage
  9. Operating Systems


Facebook RSS Twitter Twitter Google Plus


Phoronix Test Suite

OpenBenchmarking.org

Phoronix Test Suite 4.4: Blasting New Features With Forsand

Michael Larabel

Published on 23 January 2013
Written by Michael Larabel
Page 1 of 1 - 8 Comments

The first development release of Phoronix Test Suite 4.4-Forsand is now available for those interested in open-source benchmarking and automated testing. This major quarterly update is poised to introduce new features that will benefit all users from hardware enthusiasts to enterprise customers.

It's been just over one month since the release of Phoronix Test Suite 4.2-Randaberg, so the first development milestone of 4.4-Forsand has been released as the Q1'2013 update is prepared. This release doesn't exclusively focus upon improvements in one particular area but there's work going on across most areas of our open-source testing platform that powers more than one million benchmarks annually. The Git version of Phoronix Test Suite 4.4 is taking great shape and is already used in the production benchmarking at Phoronix.com.

The Phoronix Test Suite 4.4 codename is "Forsand", a city within Rogaland county, Norway. Forsand follows the codename theme of Phoronix Test Suite 4.0-Suldal and 4.2-Randaberg with being named after regions within Rogaland -- a theme that will continue with future Phoronix Test Suite 4.x releases. This builds upon the long-standing rhythm of Phoronix Test Suite 1.x codenames originating from Sør-Trøndelag, 2.x codenames from Troms, and 3.x codenames from Aust-Agder. The current plan is to release Phoronix Test Suite 4.4 officially in late February or early March while several more development "milestone" releases will be made available in the next few weeks. Among the highlights so far include:

New & Updated Tests

With OpenBenchmarking.org integration since Phoronix Test Suite 2.0-Sandtorg, test profiles and suites are no longer tied to specific versions of the Phoronix Test Suite client but remotely managed in a package-management-like solution. There's been some new test profiles released recently plus updates to other test profiles, so while they don't require you to be on Phoronix Test Suite 4.4, they are deserving of being noted.

13 New Intel Benchmarks - As written about last week, coming out of Intel's Open-Source Technology Center are 13 new test profiles covering systemd boot performance monitoring, performance testing of system libraries, and other workloads that Intel has internally used the tests for in comparing different Linux distributions.

Parboil - The Parboil software package from the IMPACT Research Group at University of Illinois has been incorporated as the newest test profile. Parboil provides OpenMP, OpenCL, and CUDA scientific tests. With the initial test profile release, selected test options include Cutoff Pair Potential, Matrix Q, MRI Gridding, Seven-Point Stencil, Lid-Driven Cavity Fluid Dynamics, and Two-Point Angular Correlation computational tests.

N-Queens - The N-Queens test profile has been updated to ensure that the compiler honors the OpenMP support.

Enhanced Error Reporting

Up to now when a test/benchmark has failed to properly install, the user would be notified and the log-file of the failed installation would be preserved. The user could then choose to dig through the log-file in hopes of seeing what's wrong -- if the upstream benchmark fails to build on a given compiler, if there happens to be some build dependency missing from the target system, or some other problem. With Phoronix Test Suite 4.4, the user will still be notified of failed installations and the log file(s) preserved, but the Phoronix Test Suite client will try to be a bit more helpful. The pts-core engine now has support for automatically and intelligently analyzing failed log files in an attempt to try to quickly determine the problem. The log files of tests that fail to properly install will be immediately scanned to look for common problems like a configre/autogen.sh script generating an error, commands not being found on the system, missing header files for tests to be compiled from source, and other common issues. A concise error string is then returned to the user at install-time about the likely problem. Enhanced error reporting will continue to be worked on through the Phoronix Test Suite 4.4 cycle and there have already been a few more improvements that landed in Git this morning for 4.4 Milestone 2.

As with all Phoronix Test Suite work, this enhanced error reporting will seamlessly work for any test written against our extensible testing architecture. Included here are screenshots showing off a few of the conveniently reported errors.

Support Checks

Phoronix Test Suite 4.4 brings a new optional feature to the test profile definitions of a "support-check" hook. Test profiles can now choose to ship a "support-check" script to analyze the target system's state prior to test installation. Up to now the support checks built into the Phoronix Test Suite have been for checking against the system's operating-system/distribution, architecture, necessary package dependencies, and other common and defined attributes. With the new "support-check.sh" handling, if a test profile ships the file, it will be called upon by pts-core to evaluate the system state. If the test profile determines the system is incompatible with the test profile, it can write its own custom error string to the file specified by the TEST_CUSTOM_ERROR environment variable that the Phoronix Test Suite sets. The Phoronix Test Suite will then deem the test not supported on the system and pass the custom error message to the user.

This extra support check has been intended to be added for a while, but was necessitated with the publicizing of the new Intel test profiles that are designed to make use of the system-provided library/state. For example, the systemd boot performance tests have support-check hooks to verify the system being tested is in fact using systemd and not another init daemon.

Phodevi

The Phoronix Device Interface (Phodevi), the multi-platform hardware/software abstraction layer used by the Phoronix Test Suite, continues to receive various fixes and other improvements. With Phodevi in Phoronix Test Suite 4.4 Milestone 1, there's improved detection of the video RAM capacity when using the Nouveau DRM driver on recent Linux kernel releases (they changed the vRAM reporting string recently), a Radeon GPU frequency detection fix for systems with integrated AMD graphics and for older graphics cards, and support for checking the GALLIUM_MSAA environment variable to see if a multi-sample anti-aliasing level was forced by user-space.

Forsand

Phoronix Test Suite 4.4 Milestone 1 also features a variety of other improvements. The enhancements include but not limited to improved external dependency handling, fallback detection for operating system lineage based upon the installed package manager, a concise public TODO list for the Phoronix Test Suite client has been made available, and the list-available-tests / list-available-suites sub-commands are now faster to display the results.

Expect Phoronix Test Suite 4.4 Milestone 2 to be released within the next two weeks. The latest release of the Phoronix Test Suite can be downloaded from Phoronix-Test-Suite.com. Commercial support, behind-the-firewall installations of OpenBenchmarking.org, and other custom enterprise services continue to be offered to organizations by contacting us, or for companies simply wishing to sponsor/support this open-source multi-platform benchmarking framework. Feature requests and other suggestions and comments can be directed to the forums.

Below is the short change-log for Forsand Milestone 1.

Phoronix Test Suite 4.4 Milestone 1
23 January 2013

- pts-core: Introduce 'common-dependencies' concept to PTS External Dependencies
- pts-core: Fix reporting of some already installed dependencies
- pts-core: Support for listing 'local/' tests from the list-supported-tests command
- pts-core: Support for test profiles providing a 'support-check.sh' file
- pts-core: Support for reporting the Graphics AA/AF values to the system notes table
- pts-core: Add a public version of the TODO list
- pts-core: Don't attempt copying install.log to install-failed.log on empty files
- pts-core: Add fallback to allow matching an operating system to an external dependencies package list based upon the detected package manager
- pts-core: When anonymous usage reporting is enabled, report test installation failures to OpenBenchmarking.org for analyzing problem
- pts-core: Support for detecting common errors within test profile install failures and reporting the relevant information to the user
- pts-core: When installing many tests and there are install failures, conveniently show the errors at the end
- pts-core: When installing multiple tests, report any unknown test profiles/suites
- pts-core: Faster loading of list-available-tests command by not forcing all test profile data be downloaded
- pts-core: Progressively download OpenBenchmarking.org in list-available-suites command to display information faster
- phodevi: Improve Nouveau video RAM memory capacity detection on newer Linux kernels
- phodevi: Warning fix for Radeon GPU frequency detection
- phodevi: Drop "special_settings_string" support
- phodevi: Support for checking if the Gallium3D anti-aliasing level was overrode using GALLIUM_MSAA environment variable

About The Author
Michael Larabel is the principal author of Phoronix.com and founded the web-site in 2004 with a focus on enriching the Linux hardware experience and being the largest web-site devoted to Linux hardware reviews, particularly for products relevant to Linux gamers and enthusiasts but also commonly reviewing servers/workstations and embedded Linux devices. Michael has written more than 10,000 articles covering the state of Linux hardware support, Linux performance, graphics hardware drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org automated testing software. He can be followed via and or contacted via .
Latest Linux Hardware Reviews
  1. MSI X99S SLI PLUS On Linux
  2. NVIDIA GeForce GTX 970 Offers Great Linux Performance
  3. CompuLab Intense-PC2: An Excellent, Fanless, Mini PC Powered By Intel's i7 Haswell
  4. From The Atom 330 To Haswell ULT: Intel Linux Performance Benchmarks
Latest Linux Articles
  1. RunAbove: A POWER8 Compute Cloud With Offerings Up To 176 Threads
  2. 6-Way Ubuntu 14.10 Linux Desktop Benchmarks
  3. Ubuntu 14.10 XMir System Compositor Benchmarks
  4. Btrfs RAID HDD Testing On Ubuntu Linux 14.10
Latest Linux News
  1. Fedora 21 Beta & Final Release Slip Further
  2. Mesa 10.3.2 Has A Couple Bug-Fixes
  3. RadeonSI/R600g HyperZ Support Gets Turned Back On
  4. openSUSE Factory & Tumbleweed Are Merging
  5. More Fedora Delays: Fedora 21 Beta Slips
  6. Mono Brings C# To The Unreal Engine 4
  7. Coreboot Now Has Support For Intel Broadwell Hardware
  8. Enlightenment's EFL 1.12 Alpha Has Evas GL-DRM Engine, OpenGL ES 1.1 Support
  9. GTK+ Lands Experimental Backend For Mir Display Server
  10. Ubuntu 14.10 Officially Released
Latest Forum Discussions
  1. AMD Radeon VDPAU Video Performance With Gallium3D
  2. HOPE: The Ease Of Python With The Speed Of C++
  3. Updated and Optimized Ubuntu Free Graphics Drivers
  4. Ubuntu 16.04 Might Be The Distribution's Last 32-Bit Release
  5. Linux hacker compares Solaris kernel code:
  6. Advertisements On Phoronix
  7. Users/Developers Threatening Fork Of Debian GNU/Linux
  8. AMD Releases UVD Video Decode Support For R600 GPUs