Better settings for the Heaven 4.0 benchmark

[Eirikr1848]

For anyone wondering why is it important to stay within Heaven's memory limits? It's because Heaven is basically testing the pure GPU processing power. If there is insufficient video memory, then the result will not reflect the "pure processing power" concept (i.e. the result will depend on the amount of installed video RAM, and the CPU-GPU texture swapping speed).

Testing GPU speed is analogous to testing CPU speed: when you do the testing, the amount of RAM shouldn't affect the results; the amount of RAM is a different concern (not the same as speed), with different consequences.

Awesome! Have you uploaded this as a separate test? I'm not sure how it works for you to be a maintainer but it sounds like you are most of the way there!

Otherwise, do you have a link to some benchmarks so we can run the test based on those settings?

Any other optimized tests youve been working on?

[Eirikr1848]

I don't have the Phoronix test suite downloaded. I'm testing Heaven on an air-gapped machine (security, you know...), so no Internet connections allowed.

I have tried over 50 different command lines for Heaven on several GPUs. I have definitively found some improvements compared to the Phoronix suite. Also found many issues.

Here are some notes:

a) I got several anomalous results, and I'm suspecting it is due to Vsync getting enabled. Use the option:
-video_vsync 0
to guarantee that vsync is disabled.

b) use the option
-console_command "render_occlusion 1 && render_refraction 1 && render_volumetric 1 && render_restart"
to guarantee that proper options are set

c) use the option
-video_multisample X
to set antialiasing level (X: 0 = 0xAA, 3 = 8xAA)

d) I think that on Windows, if Aero is disabled AND Heaven is run windowed, some anomalous results may appear, but I guess that such combination is rare.

e) I got memory requrements tabulated. Heaven 4.0 starts from minimum 260 MB VRAM (720p, 0xAA, quality low, no tess.).

f) Heaven 3.0 is also usefull. Starts from minimum 204 MB VRAM.
- it is generally slower than Heaven 4.0 (about 20% for comparable settings)
- more command line options are available: texture quality, shaders, filter, anisotropy can all be set separately.
- requires approx. 50 MB VRAM less than Heaven 4.0 to run

g) I'm now also testing Tropics 1.3. Lots of options available, but also many problems. Tropics starts at minimum 170 MB VRAM.

-------------- Heaven 4.0 quality settings ---------------------------------------------

Occlusion AA Anisotropy Filter Shaders Textures (volumetric ON)
Quality Low: off 0=off 1=2x 0=bilinear 0=low 1=med refraction off
Quality Med: on 1=2x 2=4x 1=trilinear 1=med 2=high refraction on
Quality High: on 3=8x 4=16x 1=trilinear 2=high 2=high
Quality Ultra: !!! all the same as High !!! (volumetric ON)​​

Oh ok well dang thought it could be benchmarkable with PTS and shareable on OBing.org

[mvrk]

Hi,

I fixed heaven for my installation,

i copied the test-profile for heaven from pts to local
and changed the test-definitions.xml and install.sh:

test-profiles/local/unigine-heaven/test-definition.xml
add this:

    <Option>
      <DisplayName>Quality</DisplayName>
      <Identifier>quality</Identifier>
      <Menu>
    <Entry>
          <Name>Low</Name>
          <Value>-extern_define PHORONIX,RELEASE,LANGUAGE_EN,QUALITY_LOW,TESSELLATION_EXTREME</Value>
        </Entry>
        <Entry>
          <Name>Medium</Name>
          <Value>-extern_define PHORONIX,RELEASE,LANGUAGE_EN,QUALITY_MEDIUM,TESSELLATION_EXTREME</Value>
        </Entry>
        <Entry>
          <Name>High</Name>
          <Value>-extern_define PHORONIX,RELEASE,LANGUAGE_EN,QUALITY_HIGH,TESSELLATION_EXTREME</Value>
        </Entry>
        <Entry>
          <Name>Ultra</Name>
          <Value>-extern_define PHORONIX,RELEASE,LANGUAGE_EN,QUALITY_ULTRA,TESSELLATION_EXTREME</Value>
        </Entry>
      </Menu>
    </Option>​

test-profiles/local/unigine-heaven/install.sh

change from:
./bin/heaven_x64 -data_path ../ -sound_app null -engine_config ../data/heaven_4.0.cfg -system_script heaven/unigine.cpp -video_mode -1 -extern_define PHORONIX,RELEASE \$UNIGINE_ARGS > \$LOG_FILE 2>&1

to:
./bin/heaven_x64 -data_path ../ -sound_app null -engine_config ../data/heaven_4.0.cfg -system_script heaven/unigine.cpp -video_mode -1 -video_multisample 3 -extern_plugin ,GPUMonitor \$UNIGINE_ARGS > \$LOG_FILE 2>&1

[Eirikr1848]

OK, so what you have done is the following:
- added a "Quality" option for the Heaven 4.0 benchmark
- turned ON antialiasing, always

There are a few issues:
- you have ommited the "-video_vsync 0" option. Heaven can remember the vsync settings, and it sometimes affects the results, errorneously
- I would also recommend using the "-video_fullscreen 0" option, to make results more consistent
- your modifications would make any new submission to OpenBenchmarking non-comparable to previous results
- your modifications are further increasing the number of different test modes, leading to fragmentatation of submitted results
- for the same reason as above, only a few test modes should be allowed. I have listed the recommended settings in the first post of this thread
- some people might like to test with AA off
- there is no point in offering all the possible resolution choices
- since no official on this site has responded to me so far, I guess that the admins here are not really interested in making any modifications to the Heaven 4.0 benchmark

Bro that’s because you can edit the test, just like the person did above, and like make a pull request, or you can perhaps just upload it as a separate test from within PTS.

Air gapped machine? Naw fam, come on, if you’ve got the skills to provide detailed feedback and read the test profile XML files, you’ve got the ability to submit your changes on GitHub (pretty sure it’s there, not gitlab).

That’s what contributing is all about. You got this!!!