Intel's Mesa On-Disk Shader Cache Maturing, Radeon Devs Not Yet Convinced

[marek]

I guess I'm interested in the reason if any against a proper shader cache?

I have nothing against the cache. I'd like to have the cache too. We just have to prioritize more important things at the moment.

[dungeon]

As far as I'm concerned a shader cache is needed for games like The Talos Principle.

It is not needed mostly, as actually game has its own shader cache mechanism enabled by default... So let say if your driver does not have disk cache, default is better. Otherwise, there would be a conflict there that two work similar job so if your driver have it or you enable it via driver, then disable internal one with cvar:

ogl_bDisableShaderCaching = 1

Similarly if your driver have threadedGL implemented and you enable it via driver and found that yours is let say faster, then disable internal one with cvar:

ren_bMultiThreadedRendering = 0

etc... again there is no reason two to fight to do same/similar job

[Guest]

To keep things simple we don't actually cache the IR. We just check the sha1 and either say yes we have seen this before and skip the creation of the IR or no we haven't and compile as normal. At the end of the line the driver backend also needs to cache/load its own binary format but that's pretty straight forward. The ugly part is that because we keep things simple by not storing the IR it means we have to do extra checks if we need to fallback to a full recompile rather than just using a cached IR we need to recomplie from source but this should be rare and again once its cached it wont happen again until Mesa is upgraded (in which case the cache object is removed and we fallback to a full recompile).

Sorry about inaccuracies in my comments.
It's been some time (couple months), but I seem to remember trying to wrap my head around glsl to blob serialization parts of your code and logic in glsl/linker.cpp. No need to get touchy though, way I've reviewed your code was by reading patches is chronological order.
One thing that messed with me iirc was that _some_ of early patches did more than what commit message suggested which tends to mess with me, but chances are that got fixed in subsequent RFC-worthy rebases.
Thanks for your efforts though.

[Kano]

@dungeon

Serious Engine 3.5 (Serious Sam 3) and Serious Engine 4 (The Talos Principle) precache shaders, that's correct. If you change drivers for the same card (it looks like the PCIID is used to identify) it is sometimes useful to delete the cache (gamedir/Temp/ShaderCache). But i doubt anybody will notice if the gfx driver cached it before - and those engines are definitely not affected by that change. Source 2 engine games like CS:GO seemed to stutter once a new shader was used - i guess there a cache can help much - stuttering in a competitive shooter is always bad.

[LeJimster]

I have nothing against the cache. I'd like to have the cache too. We just have to prioritize more important things at the moment.

This is what I was wondering, and it makes sense. Hopefully down the line it can be revisited.

[LeJimster]

@dungeon

Serious Engine 3.5 (Serious Sam 3) and Serious Engine 4 (The Talos Principle) precache shaders, that's correct. If you change drivers for the same card (it looks like the PCIID is used to identify) it is sometimes useful to delete the cache (gamedir/Temp/ShaderCache). But i doubt anybody will notice if the gfx driver cached it before - and those engines are definitely not affected by that change. Source 2 engine games like CS:GO seemed to stutter once a new shader was used - i guess there a cache can help much - stuttering in a competitive shooter is always bad.

Yep, this is what I'm most worried about stuttering in competitive games. I used to play fps' religiously back in the day and this would be a massive problem. Although now personally, I tend to play more casual games and still *any* stuttering just ruins immersion for me.
If we ever get to the state where VR gaming really takes off, this will need to be addressed. Luckily we aren't there yet.

[RealNC]

for ssd or hdd?

I'm using a HDD.

[dungeon]

@dungeon

Serious Engine 3.5 (Serious Sam 3) and Serious Engine 4 (The Talos Principle) precache shaders, that's correct. If you change drivers for the same card (it looks like the PCIID is used to identify) it is sometimes useful to delete the cache (gamedir/Temp/ShaderCache). But i doubt anybody will notice if the gfx driver cached it before - and those engines are definitely not affected by that change.

Yup it is same for Sam3. With amdgpu-pro user don't need to do anything else, just disable game's shader caching and multithreading, as driver already do both by default for both games

[Kano]

Well your logic is not optimal, it would still need to compile it on the fly when it is used first, then it will stutter. No idea why MT should be bad, did you do any Talos benchmarks?

[dungeon]

Yes, it is optimal - it will compile it once (as it must do that) then cache it to the disk once in the lifetime and you will never ever see it again... after that all is butter smooth

So if you are amdgpu-pro user like you isn't , do this an enjoy

ogl_bDisableShaderCaching = 1
ren_bMultiThreadedRendering = 0

No idea why MT should be bad, did you do any Talos benchmarks?

Because driver has threading profiles for both those games with amdgpu-pro and work faster with it of course, peak at plus 40% or 25% on average Something like 10 or 15% is diff (game MT vs profile MT) if i remember right and if you have enabled game MT together with profile MT it is sort of 5% slower, etc...

So the best is to disable both if you use amdgpu-pro.