RFC: A Preview Of The Phoronix Graphs With Iveland
While you should read the article if you're at all interested in benchmarking or the performance testing done by Phoronix, here's the relevant section worth repeating:
While Phoronix.com and the development of the Phoronix Test Suite are handled by separate but linked entities, of interest to many Phoronix.com readers (and the other publications that leverage this software) will be the embeddable result graphs to be found on OpenBenchmarking.org and the improvements to be found within Iveland's pts_Graph library. There has long been a desire by some to have various statistical values and other technical metrics shown within the Phoronix Test Suite result graphs found on Phoronix.com and within the Phoronix Test Suite results viewer and other places.
Starting last night and continuing through today, I have pushed the first bits of the pts_Graph overhaul into the Phoronix Test Suite Git code-base. This code is just the beginning of what's to come as far as graphing is concerned and being far from feature-complete yet for Phoronix Test Suite Iveland, which will be released in Q1'2011. However, as what's committed is already a major change visually from the existing PTS1/PTS2 graphs, here's a preview of these graphs.
As you can see already from the static version, the graphs are radically different from what you will find with pre-Iveland versions of the Phoronix Test Suite. For comparison, below is the same graph rendered by Phoronix Test Suite 2.8; it's from this week's Ubuntu Wubi benchmarks review.
The Phoronix Test Suite 3.0 graphing library adds support for horizontal bars (support for vertical bar graphs remains and can be toggled from the ~/.phoronix-test-suite/graph-config.xml along with a legacy mode). Sure to please some is also the addition of error bars being added to the bar graphs. The next two graphs are from an upcoming Intel Core i7 970 "Gulftown" Linux review, courtesy of Intel's support.
Error bars are automatically generated on relevant tests and they are the set of lines appearing on the top edge of each bar. On some graphs though it's hard to see them, as the Phoronix Test Suite already goes through a number of steps in ensuring statistical significance (and more features are to come with PTS Iveland). The standard error is also currently reported under each of the tests. Once the new pts_Graph library is finished and integrated with OpenBenchmarking.org, such information will be displayed on mouse-overs, etc.
Again, these new graphs are only the beginning and should be considered pre-alpha as they are far from being feature complete where I intend to have them at for Phoronix Test Suite 3.0 final. However, as it's already an improvement over the existing graphs, the new renderer may begin to be used at Phoronix.com in as soon as next week.
You can provide any feedback on these graphs by commenting in this forum thread (Update: There's also more graph examples and information in that thread's ongoing discussion) or emailing us.