No announcement yet.

Only use meaningful Phoronix Test Suite benchmarks in articles

  • Filter
  • Time
  • Show
Clear All
new posts

  • Only use meaningful Phoronix Test Suite benchmarks in articles

    In a comment to your article Ubuntu Intel Performance Still In Bad Shape I wrote the following:

    Originally posted by 7oby View Post
    How come that the Phoronix Testsuite now features 100+ Tests, but you happen to post the most irrelevant ones? How exactly is a KDE user or even Gnome Firefox user affected by GTK rendering performance? Please give me an example of an application that is bottlenecked by GtkRadioButton rendering performance?

    I serioulsy encourage to review the published benchmarks in terms of practical relevance. If you lack discussions of driver internals, there's not much point in posting lowlevel benchmarks and I suggest skipping that. This extends to the meaningless test of RamSpeed in different kernel versions.

    Recently a great blog of intel's performance specialist Carl Worth has been posted addressing exactly those limits of 2D rendering performance benchmarks:
    Originally posted by Carl
    Various attempts at 2D-rendering benchmark suites have appeared and even become popular. Notable examples are x11perf and gtkperf. My claim is that these tools range from useless to actively harmful when the task is understanding performance of real applications.
    (such as a GTK+ radio button)
    The punchline is that we now have an easy way to benchmark 2D rendering in actual, real-world applications. If you see someone benchmarking with only toys like x11perf or gtkperf, go ahead and point them to this post, or the the cairo-perf-trace entry in the cairo FAQ, and insist on benchmarks from real applications.

    I consider the generic Phoronix Test Suite a great tool for automated test execution and data processing of results. Therefore the test suite itself can't be blamed for including sanity tests such as x11perf and gtkperf.

    However I encourage the Phoronix team only to use real application tests whereever possible. The generic application capture tool that is proposed by Carl is only available to cairo. But maybe a similar technology will enter phoronix test suite at some point of time.

  • #2
    I was just about to post here about that tool too.

    It seems very promising, although it does not fully solve the issue you pointed -- a Qt user is as impacted by GTK rendering performance as it is by cairo rendering performance.

    The interesting thing in this case is being able to use real-world data. But yeah, I think this little tool is so promising it even deserves a little phoronix article covering it