Announcement

Collapse
No announcement yet.

Seeking Feedback On Better Visualizing Benchmark Results

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Seeking Feedback On Better Visualizing Benchmark Results

    Phoronix: Seeking Feedback On Better Visualizing Benchmark Results

    In preparation for next week's 50+ way Linux graphics comparison, I am working on some visualization improvements to better represent data from this vast assortment of graphics cards... Thus, your feedback is kindly appreciated...

    http://www.phoronix.com/vr.php?view=MTcwMjY

  • #2
    Looks nice. The new chart is visually pleasing.

    Comment


    • #3
      I've been wanting to see box plots for like forever. Ping on the pull request.

      Holy shit are there some outliers in those boxplots (I'm looking at you Intel Graphics 4600). I think what will be more helpful than the whiskers being the min and max value, is for them whiskers to be the 10th and 90th percentiles and plotting the other points as outliers, which is probably what those far out points are based on the size of the IQR. See: http://en.wikipedia.org/wiki/File:Bo...ilsabstand.png

      That way, I can say, 80% of the time I'm going to get these frame rates.

      Comment


      • #4
        I've never played with PHP and friends, so what I'm going to suggest might be impossible since I'm not aware of their limitations. Ability to change transparency/opacity of every line graph independently can be really helpful, so you can easily compare the line graphs overlapping with each other and extract more information.

        Whenever the user clicks on colored boxes or labels, a new balloon pops up and allows to change opacity value through the slider. Here is a quick and dirty mock-up:

        Comment


        • #5
          If you really want to improve the visualizations, why not make it interactive? Like these charts http://www.highcharts.com/ (sorry for the free ad, it's the closest example I've found on a quick search). You could display additional information on hover (like min/max FPS). In addition, it would be easier to find out what-is-what in benchmarks like power consumption, where results are so close you can't even distinguish lines, you could hover through the ctested cases and the correspondan line graph would have focus over the rest.

          The downside is you could no longer use images for this, of course. You would have no problem on Phoronix, for anonimous benchmarks you could use a frame or give the code in the PTS results.

          Comment


          • #6
            Originally posted by siavashserver View Post
            I've never played with PHP and friends, so what I'm going to suggest might be impossible since I'm not aware of their limitations. Ability to change transparency/opacity of every line graph independently can be really helpful, so you can easily compare the line graphs overlapping with each other and extract more information.

            Whenever the user clicks on colored boxes or labels, a new balloon pops up and allows to change opacity value through the slider. Here is a quick and dirty mock-up:
            For nearly all browsers, the pts_Graph code renders out to SVG... It'd be possible, via JavaScript, to add customizations like what you mentioned but frankly it'd be low on my TODO List unless it was for a Phoronix Premium only feature.
            Michael Larabel
            http://www.michaellarabel.com/

            Comment


            • #7
              A few things that could be improved:
              1. In the total frame time chart, you could align the min/max/avg figures across all the cards and drop the 'Min:', 'Max:' and 'Avg:' labels. It will be easier to read and cleaner too. It would be event better if the min/max/avg figures were colour coded, i.e. red for the min figure and green for the max.
              2. Entries for cards from different vendors could be coloured according to their branding colours.
              3. The text at the top, such as 'Milliseconds, Less is Better' is heavily blurred and barely readable. Consider increasing the font size or using a different font that works well at small sizes. Fonts without AA (such as Dina) generally work well at small sizes.
              4. Consider sorting the list based on performance.
              5. It would be great if the graphs were interactive and supported stable sorting based on multiple criteria.

              Comment


              • #8
                Originally posted by alazar View Post
                If you really want to improve the visualizations, why not make it interactive? Like these charts http://www.highcharts.com/ (sorry for the free ad, it's the closest example I've found on a quick search). You could display additional information on hover (like min/max FPS). In addition, it would be easier to find out what-is-what in benchmarks like power consumption, where results are so close you can't even distinguish lines, you could hover through the ctested cases and the correspondan line graph would have focus over the rest.

                The downside is you could no longer use images for this, of course. You would have no problem on Phoronix, for anonimous benchmarks you could use a frame or give the code in the PTS results.
                For most browsers already when serving OpenBenchmarking.org graphs it is served as SVG and with that is tooltips and other information that makes it a bit more interactive... But for really like rich interactivity, that's all bottlenecked by my time or the lack of code contributions by others. It's rather easy to add by anyone that has the time. Within this particular image, they were just JPEG image screenshots as I haven't uploaded the latest code to OpenBenchmarking.org for dynamic embedded purposes.
                Michael Larabel
                http://www.michaellarabel.com/

                Comment


                • #9
                  Originally posted by xslr View Post
                  A few things that could be improved:
                  1. In the total frame time chart, you could align the min/max/avg figures across all the cards and drop the 'Min:', 'Max:' and 'Avg:' labels. It will be easier to read and cleaner too. It would be event better if the min/max/avg figures were colour coded, i.e. red for the min figure and green for the max.
                  2. Entries for cards from different vendors could be coloured according to their branding colours.
                  3. The text at the top, such as 'Milliseconds, Less is Better' is heavily blurred and barely readable. Consider increasing the font size or using a different font that works well at small sizes. Fonts without AA (such as Dina) generally work well at small sizes.
                  4. Consider sorting the list based on performance.
                  5. It would be great if the graphs were interactive and supported stable sorting based on multiple criteria.
                  1. rather trivial change... patches welcome.
                  2. again, not too much of a big change, and is an easy change for new comers.
                  3. for the purposes of this article it was just JPEG screenshots rather than SVG graphs as served by most articles, until the new code is live on OpenBenchmarking.org.
                  4. for this article they were intentionally unsorted as the actual interesting data will be shared next week when it's all complete.
                  5. see my other notes in this thread and other contributions are welcome....
                  Michael Larabel
                  http://www.michaellarabel.com/

                  Comment


                  • #10
                    Originally posted by siavashserver View Post
                    I've never played with PHP and friends, so what I'm going to suggest might be impossible since I'm not aware of their limitations. Ability to change transparency/opacity of every line graph independently can be really helpful, so you can easily compare the line graphs overlapping with each other and extract more information.

                    Whenever the user clicks on colored boxes or labels, a new balloon pops up and allows to change opacity value through the slider. Here is a quick and dirty mock-up:
                    Or make it so when you hover over a box, it dims the rest of the plots.

                    Comment


                    • #11
                      Sometimes the CPU load would be essential too, especially on poorer graphics drivers. Like a video on YT is playable or game has some FPS but a lot of work is being dumped on the CPU instead of GPU. The blue FPS bar graph could have a second bar in the background with average CPU load (with high opacity) and/or CPU load timeline below FPS timeline.

                      Comment


                      • #12
                        Performance profiles

                        Performance profiles [1] can compress a huge amount of tests into a single chart with one staircase curve per test subject. It is the de facto standard in optimization benchmarks, where hundreds of tests are performed on a few test subjects. The phoronix tests are clearly different in nature, but familiarizing yourselves with this type of chart might just come in handy some day..

                        [1] http://www.optimization-online.org/D...01/03/300.html

                        Comment


                        • #13
                          - round the numbers in the bar graphs (but take account of the exact values for drawing the bars), bolden the numbers in the bar graphs,
                          - when something is “less is better”, show it in dark red instead of dark blue,
                          - dark green bars by default would be more fitting on Phoronix.

                          Comment


                          • #14
                            Originally posted by vadix View Post
                            Or make it so when you hover over a box, it dims the rest of the plots.
                            This one is easy (requires jQuery 2). Try this JSFiddle: http://jsfiddle.net/86g4r/1/

                            When you move your mouse over a box, irrelevant line graphs opacity will be greatly reduced. This will make line graphs easier to see for color blind folks too.



                            @Michael Do you accept small code contributions through JSFiddles?

                            Comment


                            • #15
                              Originally posted by siavashserver View Post
                              This one is easy (requires jQuery 2). Try this JSFiddle: http://jsfiddle.net/86g4r/1/

                              When you move your mouse over a box, irrelevant line graphs opacity will be greatly reduced. This will make line graphs easier to see for color blind folks too.
                              That is a cool improvement! I'd suggest three changes:
                              1. Shorten the animation times: 0.2 seconds for transition to near transparancy and 0.1 to opaqueness gives way quicker feedback.
                              2. Also make other lines opaque when moving over the text associated with a box. This would probably require changing the SVG to add ids to the text.
                              3. Also make the Average, Peak and Low values opaque, as these are also colored.

                              Comment

                              Working...
                              X