ATI R500: Mesa vs. Catalyst Benchmarking

While the R500 Mesa stack is fast enough for a desktop user just interested in Compiz and basic OpenGL acceleration, it isn't fast enough for performance enthusiasts or die-hard gamers. In some tests such as Warsow, Tremulous, Urban Terror, and Enemy Territory both the X1300PRO and X1800XL had playable frame-rates, the Mesa benchmarks fell significantly behind the Catalyst 8.8 driver. Well, there is an exception. The Radeon X1800XL with the Mesa git code had performed surprisingly well with the Unreal Tournament 2004 Demo and so much so that it had outperformed Catalyst 8.8 in three of the tests. It was only by a few frames, but we were certainly surprised and lead us to believe there is a regression with Unreal Tournament 2004 and the Linux Catalyst software.

With Nexuiz, the Radeon X1800XL was over twice as fast as the X1300PRO with the Mesa stack, but then the X1800XL was still left performing significantly worse than the X1300PRO using Catalyst 8.8. Nexuiz is barely playable with R500 class hardware and the current Mesa implementation. Nexuiz does offer impressive graphics and is powered by the DarkPlaces engine.

Warsow was able to deliver playable frame-rates even when running at 1920 x 1200 hardware, but this game uses the QFusion engine, which is just a modification of the Quake II engine. When switching to the Catalyst suite, both graphics cards had delivered over double the performance of Mesa.

The next open-source game tested was Tremulous and here there really wasn't too much difference between the Mesa and Catalyst drivers. Running at 1920 x 1200, Tremulous had an average frame-rate of 75 FPS when using Mesa and only 79 FPS when using Catalyst. For the Radeon X1300PRO though there was a 35 FPS difference. Tremulous is a first-person shooter with real-time strategy mix that is powered by the ioquake3 engine. With Urban Terror, which also uses the ioquake3 engine, we had experienced similar results.

In Enemy Territory with its id Tech 3 engine the results were much different from the ioquake3 games. The Radeon X1800XL with Mesa performed about 50% the speed of the Radeon X1300PRO using the proprietary driver.

The last game used for testing was the Unreal Tournament 2004 Demo. Here is where we experienced an odd issue of Mesa outperforming the Catalyst driver in three of the four tests. When running at 1280 x 1024 with ONS-Torlan, 1920 x 1200 with ONS-Torlan, and 1680 x 1050 with AS-Convoy, the Radeon X1800XL with Mesa had outperformed ATI's official driver. This was by only a few frames, but with the average frame-rate being under 40 FPS this does equate to being a few percent faster. While the X1800XL was faster, with the X1300PRO it had poorly lagged behind.

Our only synthetic tests used were in the form of the Java 2D Microbenchmark found within the Phoronix Test Suite. Here we had measured the performance for vector graphics, image rendering, and text rendering. With this Java OpenGL test, the Catalyst driver was simply no comparison to Mesa. The image rendering performance was as much as six times faster than using Mesa and with text rendering and vector graphics it was still multiple times faster.

At the end of the day, the current level of open-source 3D support for the R500 series is fine if you are just interested in using Compiz or just basic OpenGL acceleration or running a few older games. However, for any newer game you will find the frame-rate to be unbearable or not to play at all. You will notice there was no Doom 3, Quake 4, or Enemy Territory: Quake Wars benchmarks and that's because Mesa will not even work with these id Software titles. Additionally, the Unigine technology demos and Lightsmark within the Phoronix Test Suite aren't ready for Mesa graphics.

With time we suspect the performance will gradually improve, but it would require a lot of optimization work for the open-source 3D stack to be anywhere close to AMD's Catalyst / fglrx driver that has been tuned extensively and is worked on full-time by many engineers. One of the biggest open-source boosts will come when switching to the Gallium3D architecture. The open-source stack would also need to implement Catalyst AI and other advanced features for the performance to be on an even playing field.

If you're interested in comparing your system's performance to these results, it's as easy as running phoronix-test-suite benchmark michael-1567-17468-20293 after installing the Phoronix Test Suite.