Auto-Determining Relevant Linux Benchmarks To Run
Phoronix Test Suite 4.2-Randaberg Milestone 1 was released on Monday, but already there's exciting features to look forward to with the second milestone release that will happen in the near future.
The next 4.2 development release introduces a phoronix-test-suite auto-compare sub-command. When running this auto-compare command, you are prompted for what sub-system you wish to benchmark (at the moment it's mainly the CPU, disk, or GPU, but the kernel, OS, and other sub-systems will be soon committed to Git).
In terms of automatically finding relevant results for comparison, OpenBenchmarking.org searches through the hundreds of thousands of public results looking for results that are close matches to your system hardware/software but with varying components when it comes to the sub-system under test. While it may not be a 1:1 match depending upon how common your computer hardware and software is, it's fully automated and tries to find the best matches possible while the algorithms will continue to be refined prior to the 4.2-Randaberg release and the OpenBenchmarking.org continues to grow constantly, a.k.a. with time the results become organically better.
OpenBenchmarking.org also has knowledge of what component changes matter the most for a given sub-system/comparison, e.g. if trying to run a graphics comparison, finding other result files with the same processor and CPU frequency is more important than seeing there's the same network adapter or motherboard, should there be no perfect match currently in the database. Multiple passes are conducted to widen the scope when no immediate matches are detected. OpenBenchmarking.org also has some knowledge of the approximate age for how long a given hardware/software component has been on the market/available, which will be used in the future for better determining relevant result matches. Other optimizations to the detection scheme are also being evaluated and will be refined moving forward.
In addition to finding result files where the reported hardware/software are close matches to your system, it also narrows down the actual tests to execute that are useful for benchmarking the specified sub-system/component. This determination is done automatically based upon the average number of times per day that a given test profile for that sub-system is downloaded, how many other results for that given test have been uploaded to OpenBenchmarking.org, and numerous other factors to try to automatically determine the most useful and relevant tests.
In the end by simply running phoronix-test-suite auto-compare, with no prior benchmarking experience and without digging through dozens of pages on OpenBenchmarking.org looking for interesting tests and/or results, it's hopefully able to auto-determine this and then proceed with the fully-automated test installation and execution followed by results presentation.
Making things even more powerful, the auto-compare command can be seeded with specific test profiles. If you're most interested in say the server performance, you could run as an example, phoronix-test-suite auto-compare openssl apache. Seeding specific test profiles then attempts to find the close software/hardware result file matches on OpenBenchmarking.org for those tests as opposed to auto-determining tests in a particular sub-system to run.
With Phoronix Test Suite 4.2 Milestone 2 this feature is just the first-cut implementation while it will all be refined for the "Randaberg" release by year's end.
The latest Phoronix Test Suite code also introduces a phoronix-test-suite list-recommended-tests command that attempts to provides a list of "recommended" tests that are sorted on a per sub-system basis, so that one can easily see what may be the most interesting CPU or GPU tests, etc.
Similar to the test profile evaluation methods mentioned earlier in this article, the listings are automatically generated by OpenBenchmarking.org based upon the number of times each test profile has been downloaded by the Phoronix Test Suite, number of result occurrences within the public database, average results deviation, and many other factors. At the moment there's a total of 235 test profiles on OpenBenchmarking.org so this should be very useful -- particularly for new users -- in determining benchmarks to run on their given platform.
These new Phoronix Test Suite / OpenBenchmarking.org features were graciously sponsored by a Phoronix Test Suite enterprise client. If your company or organization would like to sponsor any specific features or custom engineering work on the Phoronix Test Suite or OpenBenchmarking.org, please contact Phoronix. For individuals enjoying this feature-rich cross-platform automated benchmarking framework, PayPal tips are welcome to further this project's open-source development as well as the infrastructure for the public resources provided by OpenBenchmarking.org; this development work is spearheaded at the same time as single-handedly writing hundreds of Linux articles per month. Other feedback and feature requests are welcome privately via email or within the forums.
2 Comments
