Motivation

I'd like to have an automated benchmark of a very popular game [1]. Despite poor performance ratings in Wine [2], I still find it valuable to get an idea of what performance gains can be found for a very mainstream use case.


Test profile

The test scripts will be heavily based on the excellent launcher at https://github.com/doitsujin/ffxiv-b...k-launcher.git

The graphic presets will stick to desktop presets: Standard, High or Maximum.

test-definition.xml:

Code:
<Option>
<DisplayName>Graphics Preset</DisplayName>
<Identifier>graphics-preset</Identifier>
<Menu>
<Entry>
<Name>Standard</Name>
<Value>Standard</Value>
</Entry>
<Entry>
<Name>High</Name>
<Value>High</Value>
</Entry>
<Entry>
<Name>Maximum</Name>
<Value>Maximum</Value>
</Entry>
</Menu>
</Option>
The test is downloaded from https://na.finalfantasyxiv.com/benchmark/, which I will unzip under <game path>. The executable "<game path>/game/ffxiv_dx11.exe" returns a "ffxivbenchmarklauncher.ini" file under <game path>/, containing a score and average FPS. However, I will use score as test scale:

Code:
<ResultScale>Score</ResultScale>
Filtering FPS and Score from the .ini will be straightforward, I used the following results-definition.xml:

Code:
<PhoronixTestSuite>
<ResultsParser>
<OutputTemplate>SCORE_FPSAVERAGE=#_RESULT_#</OutputTemplate>
<LineHint>SCORE_FPSAVERAGE=</LineHint>
<ResultScale>FPS</ResultScale>
</ResultsParser>
<ResultsParser>
<OutputTemplate>SCORE=#_RESULT_#</OutputTemplate>
<LineHint>SCORE=</LineHint>
<ResultScale>Score</ResultScale>
</ResultsParser>
</PhoronixTestSuite>
Not all the settings need changing for these presets. I am going to use variables for the settings in the benchmark script. The test executable will look like this:

Code:
#!/bin/bash

VIDEO_WIDTH=${1}
VIDEO_HEIGHT=${2}
VIDEO_PRESET=${3}

source ./presets_${VIDEO_PRESET}
pushd <game path>

HOME=${DEBUG_REAL_HOME} wine game/ffxiv_dx11.exe SYS.Language=1 SYS.Fps=0 SYS.ScreenMode=1 SYS.ScreenWidth=${VIDEO_WIDTH} SYS.ScreenHeight=${VIDEO_HEIGHT} SYS.FullScreenWidth=${VIDEO_WIDTH} SYS.FullScreenHeight=${VIDEO_HEIGHT} SYS.WaterWet_DX11=1 SYS.OcclusionCulling_DX11=${chk_occlusion_culling} SYS.LodType_DX11=${chk_distance_lod} SYS.ReflectionType_DX11=${cb_reflections} SYS.AntiAliasing_DX11=${chk_fxaa} SYS.TranslucentQuality_DX11=${cb_transparent_light ing} SYS.GrassQuality_DX11=${cb_grass_quality} SYS.ShadowLOD_DX11=${chk_shadow_lod} SYS.ShadowVisibilityTypeSelf_DX11=1 SYS.ShadowVisibilityTypeOther_DX11=${chk_shadow_np c} SYS.ShadowTextureSizeType_DX11=${cb_shadow_resolut ion} SYS.ShadowCascadeCountType_DX11=${cb_shadow_cascad ing} SYS.ShadowSoftShadowType_DX11=${cb_shadow_softenin g} SYS.PhysicsTypeSelf_DX11=2 SYS.PhysicsTypeOther_DX11=${grp_movement_npc} SYS.TextureFilterQuality_DX11=${cb_texture_filter_ quality} SYS.TextureAnisotropicQuality_DX11=${cb_texture_fi lter_anisotropy} SYS.Vignetting_DX11=${chk_vignette} SYS.RadialBlur_DX11=1 SYS.SSAO_DX11=${cb_ambient_occlusion} SYS.Glare_DX11=${cb_glare} SYS.DepthOfField_DX11=1 SYS.ParallaxOcclusion_DX11=${cb_parallax_occlusion } SYS.Tessellation_DX11=${cb_tessellation} SYS.GlareRepresentation_DX11=1 SYS.DistortionWater_DX11=2 >> ${LOG_FILE}

sleep 2

cat <game path>/ffxivbenchmarklauncher.ini >> ${LOG_FILE}
The file for Maximum presets, "presets_Maximum", will look like this:

Code:
chk_occlusion_culling=1
chk_distance_lod=1
chk_shadow_lod=1
chk_fxaa=1
chk_shadow_npc=1
chk_vignette=1
cb_reflections=3
cb_transparent_lighting=1
cb_glare=2
cb_shadow_resolution=2
cb_tessellation=1
cb_parallax_occlusion=1
cb_shadow_cascading=2
cb_shadow_softening=1
cb_grass_quality=3
cb_texture_filter_anisotropy=1
cb_texture_filter_quality=2
cb_ambient_occlusion=3
grp_movement_npc=2

Thoughts

On the one hand, this game seems to be very poorly optimized: for what I can see in the system monitors, the GPU utilization was not so high even during the most stressful parts.
However, it is a quite stable benchmark with very little deviation, so it does the job for me.
It's a good addition to the mix and I think represents a lot of games out there, which are also not so well optimized either.