1. Computers
  2. Display Drivers
  3. Graphics Cards
  4. Memory
  5. Motherboards
  6. Processors
  7. Software
  8. Storage
  9. Operating Systems


Facebook RSS Twitter Twitter Google Plus


Phoronix Test Suite

OpenBenchmarking.org

Auto-Determining Relevant Linux Benchmarks To Run

Phoronix

Published on 10 October 2012 01:02 AM EDT
Written by Michael Larabel in Phoronix
2 Comments

As part of a larger undertaking with OpenBenchmarking.org, the Phoronix Test Suite is now capable of intelligently and automatically determining the most relevant Linux benchmarks for you to run based upon your system's installed hardware and software along with what sub-system you're wishing to stress.

Phoronix Test Suite 4.2-Randaberg Milestone 1 was released on Monday, but already there's exciting features to look forward to with the second milestone release that will happen in the near future.

The next 4.2 development release introduces a phoronix-test-suite auto-compare sub-command. When running this auto-compare command, you are prompted for what sub-system you wish to benchmark (at the moment it's mainly the CPU, disk, or GPU, but the kernel, OS, and other sub-systems will be soon committed to Git).

Auto-Determining Relevant Linux Benchmarks To Run

With the sub-system selected, the Phoronix Test Suite then analyzes the installed system software and hardware and communicates the relevant details depending upon the sub-system to OpenBenchmarking.org. New OpenBenchmarking.org web API additions then determine the most relevant tests to run as well as result files that are relevant for merging the results and conducting a side-by-side comparison.

In terms of automatically finding relevant results for comparison, OpenBenchmarking.org searches through the hundreds of thousands of public results looking for results that are close matches to your system hardware/software but with varying components when it comes to the sub-system under test. While it may not be a 1:1 match depending upon how common your computer hardware and software is, it's fully automated and tries to find the best matches possible while the algorithms will continue to be refined prior to the 4.2-Randaberg release and the OpenBenchmarking.org continues to grow constantly, a.k.a. with time the results become organically better.

OpenBenchmarking.org also has knowledge of what component changes matter the most for a given sub-system/comparison, e.g. if trying to run a graphics comparison, finding other result files with the same processor and CPU frequency is more important than seeing there's the same network adapter or motherboard, should there be no perfect match currently in the database. Multiple passes are conducted to widen the scope when no immediate matches are detected. OpenBenchmarking.org also has some knowledge of the approximate age for how long a given hardware/software component has been on the market/available, which will be used in the future for better determining relevant result matches. Other optimizations to the detection scheme are also being evaluated and will be refined moving forward.

In addition to finding result files where the reported hardware/software are close matches to your system, it also narrows down the actual tests to execute that are useful for benchmarking the specified sub-system/component. This determination is done automatically based upon the average number of times per day that a given test profile for that sub-system is downloaded, how many other results for that given test have been uploaded to OpenBenchmarking.org, and numerous other factors to try to automatically determine the most useful and relevant tests.

In the end by simply running phoronix-test-suite auto-compare, with no prior benchmarking experience and without digging through dozens of pages on OpenBenchmarking.org looking for interesting tests and/or results, it's hopefully able to auto-determine this and then proceed with the fully-automated test installation and execution followed by results presentation.

Making things even more powerful, the auto-compare command can be seeded with specific test profiles. If you're most interested in say the server performance, you could run as an example, phoronix-test-suite auto-compare openssl apache. Seeding specific test profiles then attempts to find the close software/hardware result file matches on OpenBenchmarking.org for those tests as opposed to auto-determining tests in a particular sub-system to run.

With Phoronix Test Suite 4.2 Milestone 2 this feature is just the first-cut implementation while it will all be refined for the "Randaberg" release by year's end.

The latest Phoronix Test Suite code also introduces a phoronix-test-suite list-recommended-tests command that attempts to provides a list of "recommended" tests that are sorted on a per sub-system basis, so that one can easily see what may be the most interesting CPU or GPU tests, etc.

Similar to the test profile evaluation methods mentioned earlier in this article, the listings are automatically generated by OpenBenchmarking.org based upon the number of times each test profile has been downloaded by the Phoronix Test Suite, number of result occurrences within the public database, average results deviation, and many other factors. At the moment there's a total of 235 test profiles on OpenBenchmarking.org so this should be very useful -- particularly for new users -- in determining benchmarks to run on their given platform.

Auto-Determining Relevant Linux Benchmarks To Run

The server-side work done to OpenBenchmarking.org as part of these new features will also be exposed on the OpenBenchmarking.org web interface in the coming days. The forthcoming improvements will make it so that when viewing result files and conducting searches, the most interesting results for possible comparisons and other information will be more prominently displayed. Plus there's other features still left to be announced during this current Phoronix Test Suite development cycle.

These new Phoronix Test Suite / OpenBenchmarking.org features were graciously sponsored by a Phoronix Test Suite enterprise client. If your company or organization would like to sponsor any specific features or custom engineering work on the Phoronix Test Suite or OpenBenchmarking.org, please contact Phoronix. For individuals enjoying this feature-rich cross-platform automated benchmarking framework, PayPal tips are welcome to further this project's open-source development as well as the infrastructure for the public resources provided by OpenBenchmarking.org; this development work is spearheaded at the same time as single-handedly writing hundreds of Linux articles per month. Other feedback and feature requests are welcome privately via email or within the forums.

About The Author
Michael Larabel is the principal author of Phoronix.com and founded the web-site in 2004 with a focus on enriching the Linux hardware experience and being the largest web-site devoted to Linux hardware reviews, particularly for products relevant to Linux gamers and enthusiasts but also commonly reviewing servers/workstations and embedded Linux devices. Michael has written more than 10,000 articles covering the state of Linux hardware support, Linux performance, graphics hardware drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org automated testing software. He can be followed via and or contacted via .
Latest Linux Hardware Reviews
  1. Scythe Mugen MAX
  2. Intel Core i7 5960X Haswell-E On Linux
  3. Intel 80GB 530 Series M.2 SSD On Linux
  4. With A New Motherboard, The Core i7 5960X Haswell-E Lights Up
Latest Linux Articles
  1. 7-Way Linux Desktop Gaming Comparison On Ubuntu 14.10
  2. Intel P-State vs. CPUFreq Benchmarks On The i7-5960X
  3. RadeonSI GLAMOR Benchmarks With X.Org Server 1.16
  4. RadeonSI Gallium3D vs. Catalyst At 4K UHD On Linux
Latest Linux News
  1. AntiMicro 2.6 Yields Greater Compatibility For Gamepads On Linux
  2. OpenGL 3.3 / GLSL 3.30 Lands For Intel Sandy Bridge On Mesa
  3. AMD's RadeonSI Gallium3D Driver Sees Some Improvements
  4. Mesa 10.3 Released With The Latest Open-Source GPU Driver Improvements
  5. GNOME 3.13.92 Officially Released
  6. Wine 1.7.27 Is Still Working Towards Direct2D Support
  7. Wasteland 2 Officially Launched Today, Including For Linux Gamers
  8. Tropico 5 Launches On Steam For Linux
  9. The Intel Core i7 5960X Continues Running Great On Linux
  10. Geometry Shaders, OpenGL 3.2 Land In Mesa For Intel Sandy Bridge
Latest Forum Discussions
  1. X.Org Women Outreach Program Only Turns Up Two Applicants So Far
  2. New Group Calls For Boycotting Systemd
  3. Stop grabbing my keyboard :(
  4. ASRock AM1H-ITX: One Of The Best AM1 Mini-ITX Motherboards
  5. Glamor now enabled in Debian radeonsi
  6. Can Linux kill a motherboard?
  7. It's Now Possible To Play Netflix Natively On Linux Without Wine Plug-Ins
  8. Updated and Optimized Ubuntu Free Graphics Drivers