Phoromatic Tracker for r600g/r600c?

[runeks]

Good evening folks!

Seeing the daily (and per-commit) Phoromatic Tracker benchmarks for the kernel has made me think that it would be nice - or, dare I say, awesome even - to have something similar, a per-commit (or perhaps just daily) benchmarking, for the r600c/r600g drivers, perhaps compared to fglrx. So we'd have results like these: http://www.phoronix.com/scan.php?pag...00g_q410&num=1 only for every commit to mesa (and drm too?) - if resources allow it, if not then only daily.

xorg-edgers updated their packages a couple of days ago, and I just ran a new benchmark of nexuiz and the FPS has increased by 5.5% since the last version of the drivers:

http://global.phoronix-test-suite.com/index.php?k=profile&u=runeks-7001-25492-1244

I just think it'd be really cool to have this functionality in the Tracker. Are there any reasons - technical, financial, or otherwise - that this hasn't been done yet? One I can think of is the energy usage, which I suspect would be quite a bit higher - ie. Radeoon HD 4870 vs. an intel Atom CPU.

[Michael]

Yeah, basically there is no daily high-end GPU testing because 1. energy cost of running a high-end GPU all the time for testing 2. to a lesser extent, noise level of more powerful systems running constantly in my office.

There's also the matter that it's harder to track the performance of a single graphics component of the open-source stack since sometimes Mesa requires updated DRM/libdrm, etc. So it would be a matter of tracking multiple components which would make it a bit more difficult to effectively find any regressions, but yeah, mostly comes down to the first two issues.

[runeks]

I see. I suspected as much. Just wanted to make sure it had been thought of (of course it had ).

Still, even though per-commit or per-day benchmarking would be too energy-consuming, it would be really nice to put the results of, for example, the aforementioned article in scope. Ie.: Nexuiz@1024x768 with r600g runs at 42.03 FPS on an HD4870, but what was this performance parameter one/two/six months ago? Has it gotten better, or worse? And how has r600g evolved compared to r600c? That would be really interesting if you ask me.

Although I'm perhaps picking out the wrong article, as it seems to be about r600c/r600g vs. fglrx, it'd be nice to have an overview of the general performance of 3D on Linux over time as a whole (ie. including DRM improvements etc.).

Have you considered doing weekly benchmarks of a setup with the most recent r600c/r600g/DRM/libdrm components from git, and putting that up somewhere? Showing the most recent performance figures in relation to historical ones.

[curaga]

How 'bout a passive 5570

[runeks]

Or you could slap an Arctic Cooling Accelero S1 Rev. 2 on your 4870. It works pretty well on my 3870. It's as huge as a friggin book, but I guess that's necessary. Idles at ~54C.

[Michael]

Again, the bigger issue is the excess energy consumption.

[runeks]

Again, the bigger issue is the excess energy consumption.

OK. And you have no plans to run, for example, weekly benchmarks?

Maybe it's just me that has a thing with the evolution of these drivers over time, but I think it would be neat to have a general overview of the progress over time. Weekly benchmarks would probably be enough anyway, I don't expect the performance to change that much over time. But I must say I was pretty impressed when I updated to the xorg-edgers PPA from the Lucid stock drivers. A Nexuiz 800x600 test went from 2.94 FPS to 43.05 FPS. That's pretty amazing!

[Michael]

OK. And you have no plans to run, for example, weekly benchmarks?

Maybe it's just me that has a thing with the evolution of these drivers over time, but I think it would be neat to have a general overview of the progress over time. Weekly benchmarks would probably be enough anyway, I don't expect the performance to change that much over time. But I must say I was pretty impressed when I updated to the xorg-edgers PPA from the Lucid stock drivers. A Nexuiz 800x600 test went from 2.94 FPS to 43.05 FPS. That's pretty amazing!

I won't do a Phoromatic Tracker that isn't easily fully automated already or where it's on a weekly basis for turning it on, etc. Maybe at a later point when I can more easily justify the expense, but not at the moment.

[runeks]

I won't do a Phoromatic Tracker that isn't easily fully automated already or where it's on a weekly basis for turning it on, etc. Maybe at a later point when I can more easily justify the expense, but not at the moment.

OK, cool. Thanks for clarifying.

I will be doing some completely un-automated benchmarks, on a whenever-I-feel-like-and-remember-it basis, of the drivers from xorg-edgers on Phoronix Global, in case anyone's interested. Here's the latest: http://global.phoronix-test-suite.co...14-26268-31062

[Michael]

I will be doing some completely un-automated benchmarks, on a whenever-I-feel-like-and-remember-it basis, of the drivers from xorg-edgers on Phoronix Global, in case anyone's interested. Here's the latest: http://global.phoronix-test-suite.co...14-26268-31062

With PTS3-Iveland and OpenBenchmarking.org you'll be able to do lots cooler stuff too where it will be of greater value