Heaven 4.0 benchmark - suggested improvements

[drastic]

Michael

Hello!

I have noticed a problem with your Heaven 4.0 benchmark on openbenchmarking.org .

The benchmark settings that you are using are non-standard, non-comparable to other results on the Internet, and those settings do not produce "sufficiently fair" results across architectures and generations of GPUs. There also a few other issues. I really like the Heaven benchmark otherwise, so I thought I could help there.

I can post the correct command lines for multiple benchmark settings, if you are interested. But it will be a long post, and requires some work from my side, so I would like a confirmation from you that you are interested (i.e. that you will at least read/consider what I write).

Are you interested?

[drastic]

This is the best I could figure out:
The first issue is that other websites are testing with the TESSELLATION_EXTREME setting, but openbenchmarking.org doesn't specify the tessellation setting, so it defaults to TESSELLATION_NORMAL.
However, that should only give 15% higher scores, but openbenchmarking.org results are higher 30% than expected (inconsistently).

I don't know where the remaining difference in FPS results comes from.

[drastic]

Oh, I got it.
Openbenchmarking omits the parameter
-project_name "Heaven"
That disables antialiasing. Since some GPUs are better at antialiasing, and some are worse, it produce s inconsistencies compared to results from other websites.

I think that testing with antialiasing=off will not produce representative results. Everyone has been using antialiasing for at least a few years.

[drastic]

I just tested it and I can confirm that you can get back antialiasing by adding the parameter:
-video_multisample 3

To get extreme tesselation, use:

"PHORONIX,RELEASE,QUALITY_ULTRA,TESSELLATION_EXTREME"

Also, I think that not setting video-mode is a really bad idea for many reasons.
For 1280x720, use: -video_mode 3
For 1600x900, use: -video_mode 5
For 1920x1080, use: -video_mode 6

And I really think that fullscreen should not be allowed, for many many reasons, so that option should be disabled.

[drastic]

And I got many other details added here (but it is a long post):

https://www.phoronix.com/forums/forum/phoronix/phoronix-test-suite/1413808-better-settings-for-the-heaven-4-0-benchmark

[drastic]

Michael
I got you the solution, two posts above, if you are interested. Just a ping.