Valve + LunarG Open Up Their Mesa Testing Results

[phoronix]

Phoronix: Valve + LunarG Open Up Their Mesa Testing Results

As covered back during XDC2017, Valve and LunarG have been working on more extensive testing of Mesa to catch regressions and meticulously spot any performance changes as they occur. That framework is now publicly available to see the results and for developers allows tracking their own Mesa development branches...

http://www.phoronix.com/scan.php?page=news_item&px=Valve-LunarG-Share-Results

[stalkerg]

I think it's better than nothing. At least we can find regressions quickly.

[microcode]

I think it's better than nothing. At least we can find regressions quickly.

The problem is that the signals will be false, because you can have a big boost in the speed of playing back traces, while actually decreasing throughput for the application those traces are from. Fundamentally, I think you really have to run the whole application if at all possible.

[microcode]

Phoronix: Valve + LunarG Open Up Their Mesa Testing Results

As covered back during XDC2017, Valve and LunarG have been working on more extensive testing of Mesa to catch regressions and meticulously spot any performance changes as they occur. That framework is now publicly available to see the results and for developers allows tracking their own Mesa development branches...

http://www.phoronix.com/scan.php?pag...-Share-Results

I keep wondering when they'll just contract Michael Larabel to run a big dirty regression test cluster. I'm trying to put something similar together for SANE, getting a bunch of scanners in to do regression testing on the scanner drivers.

[ua=42]

The problem is that the signals will be false, because you can have a big boost in the speed of playing back traces, while actually decreasing throughput for the application those traces are from. Fundamentally, I think you really have to run the whole application if at all possible.

Ummm... No.
While the results won't indicate how fast the game actually will run... it is a good test setup for testing GPU issues.

[microcode]

Ummm... No.
While the results won't indicate how fast the game actually will run... it is a good test setup for testing GPU issues.

Good for testing, bad for comparison. The fundamental problem is that your system has two free variables (the platform and the application), whereas an OpenGL to OpenGL comparison has only one free variable (the platform). You have no way of distinguishing problems with the platform from problems with the application.

[Holograph]

The problem is that the signals will be false, because you can have a big boost in the speed of playing back traces, while actually decreasing throughput for the application those traces are from. Fundamentally, I think you really have to run the whole application if at all possible.

This doesn't seem to be logical to me, though I could be wrong. The speed of something is the inverse of the sum of the latency of every step. Any latency you reduce in the process speeds up the overall process. And if this is used to track Mesa, well, Mesa is only used for video stuff, and video stuff is what they are testing. I don't think there's ANY (literally any) risk of this indicating a speedup in a situation that actually slows down in the real world. I think the only potential problem here is that they might indicate a 10% speedup for their testing, only for the real world speedup to be more like 2% when factoring in the rest of the game code.

I COULD BE WRONG; this is just what sounds logical to me based on the article.

[microcode]

This doesn't seem to be logical to me, though I could be wrong. The speed of something is the inverse of the sum of the latency of every step. Any latency you reduce in the process speeds up the overall process. And if this is used to track Mesa, well, Mesa is only used for video stuff, and video stuff is what they are testing. I don't think there's ANY (literally any) risk of this indicating a speedup in a situation that actually slows down in the real world. I think the only potential problem here is that they might indicate a 10% speedup for their testing, only for the real world speedup to be more like 2% when factoring in the rest of the game code.

I COULD BE WRONG; this is just what sounds logical to me based on the article.

There are lots of cases where you could indicate a speedup where a slowdown is what happens in reality. Computers have caches, and caches are a limited shared resource. When your application calls into the graphics driver, the graphics driver has to share the L1/L2/L3 caches with the application. In practice, this means that if the driver makes a tradeoff of more memory accesses for greater throughput, it could speed up the trace, but slow down the real world application, because in the actual application, there is more pressure on the caches, and the driver's additional memory usage causes the application's cache lines to be evicted.

There are other ways in which trace performance and real application performance may differ in opposite directions, but this should convince you that it is possible. As for how likely this is to be a problem, I can't say.

[yoshi314]

[quote}If graphics driver developers need access to the traces so they can debug/profile a change, Valve is providing them directly to the developers on a case-by-case basis after granting them a gratis copy of the particular game.[/quote]
this may cause an influx of "developers". just like it was with that Debian giveaway.