Interestingly enough, minecraft would make a good automated benchmark because anyone can have the client download x version of the game. They built in the ability to download old versions. Also, the game can be launched via command line. It is also one of the most popular games of all time and quickly heading for the number 1 spot, which means it is relevant to allot of gamers. It would be interesting to see how well it performs on the open-source drivers.
But, if you are interested, they have a page where people load the same world, at the same resoltuion, with the same settings and post their performance there. There are several linux entries there if you are curious.
But, if you are interested, they have a page where people load the same world, at the same resoltuion, with the same settings and post their performance there.
Sadly this isn't enough. Things like mob AI (pathfinding, target selecting, ...), weather and other things are based on random generators. You would have to modify Minecraft to initialize all these generators with the same seeds every time but as Notch is a bad Java programme that might be a lot of work (I mean: protected int k = (new Random()).nextInt(); - seriously? In the same file we have public Random random = new Random(); - but that's just the server, we need the client, too...).