Thanks, Michael Larabel for continuing to post meaningless benchmarks and drawing wrong conclusions.
http://keithp.com/blogs/Sharpening_t..._Driver_Focus/Originally Posted by Keith Packard
But as Keith already mentions: glxgears is not a benchmark. But maybe you need one more:
http://www.cworth.org/intel/performance_measurement/Originally Posted by Carl Worth
Now that we laughed about you and Michael for drawing conclusions from this same benchmark (just ported to QT4):
Let's consider some serious aspects. We need real application benchmarks such as game engines, firefox rendering or compiz performance. But even in this case, performance may degrade from one version to another. If that's the reason one should check whether one of these reasons apply:
. I noticed in Urban Terror, when performing Mesa benchmarks, that in certain revisions of the Mesa Stack on x3100 GM965 hardware, some visual effects were not drawn e.g. some lightning or the shot of your weapon. If it's not drawn the entire scene draws faster (though with slight visual corruption or flaws) and you get higher fps reports.
. In some of the recent versions of the intel driver tearing disappeared. I'm not aware of in which version exactly and whether it applies only to indirect or direct rendering, but this might may a dfficerence. I remember Jesse talking about double buffering:
I don't know whether this entered mainline or whether it's related. BUT: I don't see tearing with 2.8.0 in compiz enabled X-Org anymore and I know that double buffering has an impact on reported framerates (compared to tearing withough v-sync):
Michael, please remove QGears2 benchmarks from the benchmark suite that you use for performance evaluation. That applies to the GTK benchmarks as well that you used in this article, which I critized already some time ago:
This was the case with 9.04 for example: the final build had pretty much the same performance as the alphas, at least on my hardware (Ati R500 with the OSS drivers).
Is ubuntu 9.10 going to be faster then 9.04 for intel users?
You answer: We tested it with phoronix-test-suite and our conclusion is that the performance will be the same but slightly more unstable. But we tested the newest intel drivers and rc kernel on another distro (like arch, gentoo etc) and it showed boost in performance so we can't be sure at this point.
I untill recently was very dissapointed in intel's performance although it has advantages over other gpu's because I can built it straight into kernel and I don't have to worry about will nVIDIA or ATI release a driver that will compile on this rc kernel. I like oss and that is one of many reasons I use linux. But now, when they finally did a decent job I read more dissapointing news here on phoronix... Maybie those guys @canonical are doing something wrong but as I say 2.6.31+2.8.0 works perfectly stable and as fast as intel drivers ever where...
In general, if you look at some of the "this made some one look really bad" articles, the group with the gap go away, do a few blog postings and discuss what happened. This happened with the SuSE barrier usage. It caused discussion, it caused awareness.
Phoronix is just taking a mid-stream alpha and benchmarking it.
If the numbers are wrong, do a PTS response to show where the numbers are wrong.
If the numbers are right, work with the Ubuntu community to close the gap, offer to help Bryce select the right patch set for the intel drivers. You only have a few milestones left.
If you don't believe the tests are valid, help define some valid ones to improve the quality of the information.
Ok here are my results:
gtkperf: 2.6.30 - 1676 ; 2.6.31-rc4 - 1573
glmark: 2.6.30 - 64 ; 2.6.31-rc4 - 70
urban terror: 2.6.30 - 20.35 ; 2.6.31-rc4 - 23.70
So I don't think that there is regression on 2.6.31-rc just an improvement
It's not a question of the phoronix test suite itself, but of the benchmarks choosen. If developers itself need profiling software to recognize which software pieces are used and to detect software hot spots, what do you think what kind of tools a reviewer needs?
Well, Eric Anholt needs cairo-perf-trace:
http://anholt.livejournal.com/41306.htmlThanks to cairo-perf-trace, I've just landed a 10% improvement in firefox performance for UXA
Phoronix needs to recognize that it's low level synthetic benchmarks are meaningless. It needs to focus on real applications, because it's simply not experienced enough to work with other benchmarks in order to draw the right conclusions.
Benchmarking determines a macro issue that needs attention, profiling allows developers to deep dive into hotspots that need attention and provide improvements in the benchmark.
Note that cairo-perf-trace generates an application call profile, which then can be played as a benchmark. The developers then profile the benchmark to get the hotspots.
As has been mentioned before, if people are concerned about the quality or correctness of the benchmarks themselves, by all means, invest the effort into getting cairo-perf-trace into PTS. Be part of the solution .Phoronix needs to recognize that it's low level synthetic benchmarks are meaningless. It needs to focus on real applications, because it's simply not experienced enough to work with other benchmarks in order to draw the right conclusions.
A final comment...
I don't think Phoronix pulls many conclusions other than the benchmarks presented show a performance delta. What does happen though is the parties that are implicated by the performance issue actually begin to communicate, an example is
Bryce (Canonical) concurred about a potential performance issue associated with the intel drivers. He raises a bug to track, intel acknowledges and looks to reproduce the issue. The critical part is that a DISCUSSION is now underway to quantify if there is an issue.
That is the value of the Phoronix benchmark articles. It raises to the surface potential issues and triggers a discussion that would most likely not have occured in the past.