Mesa Shader Compiler Cache Proposed, Reduces Game Start Times

[przemoli]

Any improvements on non-Steam, free/libre games? I doubt the graphic driver matters there.

Still plenty of games that use shaders.

Currently, all those shaders are compiled every time they need to be used (unless game keep track of compiled shaders, but then linking on recombining them still take time), recent changes to Mesa that landed added in-RAM cache for it (so game do not need to keep track, but still on restart of OS, cache is lost), now we will get persistent cache.

All of that is very, very good. Especially on single threaded apps, where shader compilation stalled all the other jobs.

[zoomblab]

Is this for all drivers or only for intel? Will it work automatically?

[whitecat]

Is this for all drivers or only for intel? Will it work automatically?

This patchset applies to the GLSL compiler so this include the Gallium drivers.
But currently this patchset is broken for Gallium. The author is trying to fix it.

AFAIK, in order to work, the game must explicitly tells the driver to cache the shaders. This is the case for Left 4 Dead 2 for instance. Some games, like ETQW, Xonotic, 0AD, etc., doesn't do that.

[smitty3268]

AFAIK, in order to work, the game must explicitly tells the driver to cache the shaders. This is the case for Left 4 Dead 2 for instance. Some games, like ETQW, Xonotic, 0AD, etc., doesn't do that.

That's how you use the GL extension (ARB_get_program_binary), which is part of GL4.1. This patchset, however, does add an automatic cache that's used regardless of whether the apps ask for it or not, and isn't standard GL at all - just something extra that's easy to add on once you are able to do so. NVidia's driver does the same thing.

[whitecat]

That's how you use the GL extension (ARB_get_program_binary), which is part of GL4.1. This patchset, however, does add an automatic cache that's used regardless of whether the apps ask for it or not, and isn't standard GL at all - just something extra that's easy to add on once you are able to do so. NVidia's driver does the same thing.

When I launch these games, there is no cache created.

[smitty3268]

When I launch these games, there is no cache created.

It looks like something is wrong, then.

From the patch author:

This series provides a disk cache for the shader compiler and is used
to 'skip glLinkProgram' like GL_ARB_get_program_binary does but under
the hood without api for the client.

Looking at the code, i see in patch 18:

int
+mesa_program_diskcache_init(struct gl_context *ctx)
+{
+ const char *tmp = "/tmp", *cache_root = NULL;
+ int result = 0;
+
+ cache_root = _mesa_getenv("XDG_CACHE_DIR");
+ if (!cache_root)
+ cache_root = _mesa_getenv("HOME");
+ if (!cache_root)
+ cache_root = tmp;
+
+ ctx->BinaryCachePath = ralloc_asprintf(ctx, "%s/.cache/mesa", cache_root);
+
+ struct stat stat_info;
+ if (stat(ctx->BinaryCachePath, &stat_info) != 0)
+ result = mesa_mkdir_cache(ctx->BinaryCachePath);
+
+ if (result == 0)
+ ctx->BinaryCacheActive = true;
+ else
+ ctx->BinaryCacheActive = false;
+
+ return result;
+}

And then patch 19 goes on to call that function, and use the cache if the ctx->BinaryCacheActive is set to true.

Maybe your application can't access whatever directory it's trying to use?

[V10lator]

When I launch these games, there is no cache created.

Definitely something wrong then. I tried the patches with r600g some days ago and while many 3d applications crashed (Cairo-Dock, Steam, compton, ... sigsegv somewhere in r600g_dri.so), did a 2d fallback (xfce4-terminal, xfce4-panel, ...) or rendered garbage (Chromium) at least it created a cache:

$ ls ~/.cache/mesa
12193688422293999815.bin  13610198304194110691.bin  349158879344228341.bin
12193688423394689806.bin  1361019830831174043.bin   35282466041955907973.bin
13328931782507786131.bin  1389218944831174043.bin   495130997817265084.bin
13328931783394689806.bin  3218773652508479836.bin

Please note that I didn't even start a game, just the desktop and Chromium.

[whitecat]

Definitely something wrong then. I tried the patches with r600g some days ago and while many 3d applications crashed (Cairo-Dock, Steam, compton, ... sigsegv somewhere in r600g_dri.so), did a 2d fallback (xfce4-terminal, xfce4-panel, ...) or rendered garbage (Chromium) at least it created a cache:

$ ls ~/.cache/mesa
12193688422293999815.bin  13610198304194110691.bin  349158879344228341.bin
12193688423394689806.bin  1361019830831174043.bin   35282466041955907973.bin
13328931782507786131.bin  1389218944831174043.bin   495130997817265084.bin
13328931783394689806.bin  3218773652508479836.bin

Please note that I didn't even start a game, just the desktop and Chromium.

You can try the l4d2_testing branch, it fix the Steam crash.

http://cgit.freedesktop.org/~tpalli/mesa/log/?h=l4d2_testing

[whitecat]

Maybe your application can't access whatever directory it's trying to use?

I don't think so. The cache is stored on HOME for me (I haven't the XDG_CACHE_DIR env var).
Probably something is wrong since the author said it is not tested on gallium driver.

[V10lator]

You can try the l4d2_testing branch, it fix the Steam crash.
http://cgit.freedesktop.org/~tpalli/...h=l4d2_testing

Maybe I'll try that later but my guess is that it might still not work. I think gallium and/or the r600g driver need some changes. Anyway with testing this I might be able to give Tapani P?lli a gdb backtrace which I think he still needs (I can't find any reply from Benjamin after that mail: http://lists.freedesktop.org/archive...ne/060663.html ), so I might really do it when I find the time for.