Better settings for the Heaven 4.0 benchmark

[drastic]

Here is my attempted fix for the OpenBenchmarking Tropics 1.3 test (problems were outlined in the previous post):

Tropics.exe   -video_app direct3d9   -sound_app null   -video_vsync 0   -video_multisample 0     -data_path ./    -engine_config ../data/unigine.cfg    -system_script tropics/unigine.cpp   -extern_define PHORONIX,RELEASE    -console_command "render_shaders 2   &&   render_textures 2  &&    render_filter 1  &&   render_anisotropy 2  &&   render_occlusion 1   &&   render_reflection 1  &&   render_refraction 1   &&   render_volumetric 1  &&  render_restart"
$UNIGINE_ARGS

uniginine_args then adds the following:
-video_width 1920 -video_height 1080 -video_fullscreen 1

In my opinion, the test would also be much better with antialiasing enabled and anisotropy at 8x or 16x, and the number of available modes should be reduced (to allow only a few preset modes, and those presets should be similar to presets in Heaven 4.0; the last one below is the "original" mode from Tropics). The modes should be:

1920x1080 "Legacy" - same settings as the ones used now: 0xAA, 4xAF, but in DirectX 9
1920x1080 "High FHD"- same settings as in the Valley 1.0 preset "Extreme FHD", DirectX 9 (all settings maxed out)
1600x900 "High"- same settings as in the Heaven 4.0 preset "Extreme", DirectX 9 (all settings maxed out)
1280x720 "Basic"- same settings as in the Heaven 4.0 preset "Basic", DirectX 9
1024x768 "Original Small"- original preset from Tropics 1.3: AA off, shaders high, textures high, trilinear, 4x, occlusion OFF, reflection ON, refraction ON, volumetric ON, DirectX 9

Edit:
I would also take the opportunity to add this mode:
2560x1440 "QHD 4xAA"- higher resolutions require less anti-aliasing, but all other settings maxed out.

I would also like to add that Tropics 1.3 looks beautifull in ultra-widescreen resolutions like "Ultrawide" 2560×1080, so I suggest to everyone to try it out. It easily does over 60 FPS on almost any modern GPU. You will have to use the command-line to set the resolution, and also don't forget to enable vsync in the command-line.

On Linux, you have to use the OpenGL renderer, which will likely be limited to 60 FPS. Works on Ubuntu and derivatives (don't enable vsync).

To add the sound on Windows, download OpenAL-soft:
### https://openal-soft.org/openal-binar...1.22.2-bin.zip
in it, find the file "bin/Win32/soft_oal.dll", rename it to "openal32.dll" and place it next to the Tropics.exe , and add the sound in the command-line (openal instead of null )

[Eirikr1848]

I don't have the Phoronix test suite downloaded. I'm testing Heaven on an air-gapped machine (security, you know...), so no Internet connections allowed.

I have tried over 50 different command lines for Heaven on several GPUs. I have definitively found some improvements compared to the Phoronix suite. Also found many issues.

Here are some notes:

a) I got several anomalous results, and I'm suspecting it is due to Vsync getting enabled. Use the option:
-video_vsync 0
to guarantee that vsync is disabled.

b) use the option
-console_command "render_occlusion 1 && render_refraction 1 && render_volumetric 1 && render_restart"
to guarantee that proper options are set

c) use the option
-video_multisample X
to set antialiasing level (X: 0 = 0xAA, 3 = 8xAA)

d) I think that on Windows, if Aero is disabled AND Heaven is run windowed, some anomalous results may appear, but I guess that such combination is rare.

e) I got memory requrements tabulated. Heaven 4.0 starts from minimum 260 MB VRAM (720p, 0xAA, quality low, no tess.).

f) Heaven 3.0 is also usefull. Starts from minimum 204 MB VRAM.
- it is generally slower than Heaven 4.0 (about 20% for comparable settings)
- more command line options are available: texture quality, shaders, filter, anisotropy can all be set separately.
- requires approx. 50 MB VRAM less than Heaven 4.0 to run

g) I'm now also testing Tropics 1.3. Lots of options available, but also many problems. Tropics starts at minimum 170 MB VRAM.

-------------- Heaven 4.0 quality settings ---------------------------------------------

Occlusion AA Anisotropy Filter Shaders Textures (volumetric ON)
Quality Low: off 0=off 1=2x 0=bilinear 0=low 1=med refraction off
Quality Med: on 1=2x 2=4x 1=trilinear 1=med 2=high refraction on
Quality High: on 3=8x 4=16x 1=trilinear 2=high 2=high
Quality Ultra: !!! all the same as High !!! (volumetric ON)​​

Oh ok well dang thought it could be benchmarkable with PTS and shareable on OBing.org

[drastic]

Oh ok well dang thought it could be benchmarkable with PTS and shareable on OBing.org

I estimate that most people reading Phoronix are not even aware that the PTS Heaven test is conducted with antialiasing off, and that it is a non-standard setting. This option is not specified anywhere, it just happens that Heaven defaults to disabling antialiasing when no "project name" is provided.

What I was suggesting is a way to make the Heaven 4.0 (and Tropics) test produce more meaningfull results (by using their built-in standard presets). Then you can compare those results to other websites, for example:
### https://www.overclockersclub.com/rev...r9_270x/12.htm
### https://www.techpowerup.com/forums/t...part-2.222125/

[mvrk]

Hi,

I fixed heaven for my installation,

i copied the test-profile for heaven from pts to local
and changed the test-definitions.xml and install.sh:

test-profiles/local/unigine-heaven/test-definition.xml
add this:

    <Option>
      <DisplayName>Quality</DisplayName>
      <Identifier>quality</Identifier>
      <Menu>
    <Entry>
          <Name>Low</Name>
          <Value>-extern_define PHORONIX,RELEASE,LANGUAGE_EN,QUALITY_LOW,TESSELLATION_EXTREME</Value>
        </Entry>
        <Entry>
          <Name>Medium</Name>
          <Value>-extern_define PHORONIX,RELEASE,LANGUAGE_EN,QUALITY_MEDIUM,TESSELLATION_EXTREME</Value>
        </Entry>
        <Entry>
          <Name>High</Name>
          <Value>-extern_define PHORONIX,RELEASE,LANGUAGE_EN,QUALITY_HIGH,TESSELLATION_EXTREME</Value>
        </Entry>
        <Entry>
          <Name>Ultra</Name>
          <Value>-extern_define PHORONIX,RELEASE,LANGUAGE_EN,QUALITY_ULTRA,TESSELLATION_EXTREME</Value>
        </Entry>
      </Menu>
    </Option>​

test-profiles/local/unigine-heaven/install.sh

change from:
./bin/heaven_x64 -data_path ../ -sound_app null -engine_config ../data/heaven_4.0.cfg -system_script heaven/unigine.cpp -video_mode -1 -extern_define PHORONIX,RELEASE \$UNIGINE_ARGS > \$LOG_FILE 2>&1

to:
./bin/heaven_x64 -data_path ../ -sound_app null -engine_config ../data/heaven_4.0.cfg -system_script heaven/unigine.cpp -video_mode -1 -video_multisample 3 -extern_plugin ,GPUMonitor \$UNIGINE_ARGS > \$LOG_FILE 2>&1

[drastic]

Hi,
I fixed heaven for my installation ...

OK, so what you have done is the following:
- added a "Quality" option for the Heaven 4.0 benchmark
- turned ON antialiasing, always

There are a few issues:
- you have ommited the "-video_vsync 0" option. Heaven can remember the vsync settings, and it sometimes affects the results, errorneously
- I would also recommend using the "-video_fullscreen 0" option, to make results more consistent
- your modifications would make any new submission to OpenBenchmarking non-comparable to previous results
- your modifications are further increasing the number of different test modes, leading to fragmentatation of submitted results
- for the same reason as above, only a few test modes should be allowed. I have listed the recommended settings in the first post of this thread
- some people might like to test with AA off
- there is no point in offering all the possible resolution choices
- since no official on this site has responded to me so far, I guess that the admins here are not really interested in making any modifications to the Heaven 4.0 benchmark

[Eirikr1848]

OK, so what you have done is the following:
- added a "Quality" option for the Heaven 4.0 benchmark
- turned ON antialiasing, always

There are a few issues:
- you have ommited the "-video_vsync 0" option. Heaven can remember the vsync settings, and it sometimes affects the results, errorneously
- I would also recommend using the "-video_fullscreen 0" option, to make results more consistent
- your modifications would make any new submission to OpenBenchmarking non-comparable to previous results
- your modifications are further increasing the number of different test modes, leading to fragmentatation of submitted results
- for the same reason as above, only a few test modes should be allowed. I have listed the recommended settings in the first post of this thread
- some people might like to test with AA off
- there is no point in offering all the possible resolution choices
- since no official on this site has responded to me so far, I guess that the admins here are not really interested in making any modifications to the Heaven 4.0 benchmark

Bro that’s because you can edit the test, just like the person did above, and like make a pull request, or you can perhaps just upload it as a separate test from within PTS.

Air gapped machine? Naw fam, come on, if you’ve got the skills to provide detailed feedback and read the test profile XML files, you’ve got the ability to submit your changes on GitHub (pretty sure it’s there, not gitlab).

That’s what contributing is all about. You got this!!!

[drastic]

Bro that’s because you can edit the test, just like the person did above, and like make a pull request, or you can perhaps just upload it as a separate test from within PTS.

Air gapped machine? Naw fam, come on, if you’ve got the skills to provide detailed feedback and read the test profile XML files, you’ve got the ability to submit your changes on GitHub (pretty sure it’s there, not gitlab).

That’s what contributing is all about. You got this!!!

I can 'contribute' the code for the files, that is not an issue. The issue is that I cannot test it.

So, if you can find someone who can test my contributions, I'll be glad to post them here. Probably, the number of bugs will be low, because the changes are simple.

When the tester finds that my changes are acceptable, he can make the pull request. Also, if I make some small errors, he should fix them.

Is that acceptable? I'm willing to post the improvements here if you can find me a tester. Also, my security policy is my decision to make; critiques on that topic will be ignored.