Mesa Turns To BLAKE3 For Faster Vulkan Shader Hashing

[phoronix]

Phoronix: Mesa Turns To BLAKE3 For Faster Vulkan Shader Hashing

Mesa has switched from SHA1 to BLAKE3 for its shader hashing to deliver better performance...

https://www.phoronix.com/news/Mesa-BLAKE3-Shader-Hashing

[Archprogrammer]

Nice to see that people start thinking about the properties of hashes instead of just using SHA256 (or MD5) just because it's the first one that comes to mind. Protecting from attacks isn't the point here, so you want a fast hash, not a slow one.

[Quackdoc]

i've been using b3sum for a while now for manually hash checking, it's quite fast which is really nice

[arzeth]

Nice to see that people start thinking about the properties of hashes instead of just using SHA256 (or MD5) just because it's the first one that comes to mind. Protecting from attacks isn't the point here, so you want a fast hash, not a slow one.

Yes, but it still should be a cryptographically secure hash, just not necessarily very strong or time-tested. The fastest non-cryptographically-secure hash algo is xxHash which is 10x faster than BLAKE3, but it's very easy to get collisions. WMs/compositors use shaders (probably) to display a window's shadow, let's say this shader's hash is 0xABC. Now imagine:
1. An upcoming version of the WM switches to a new shadow shader whose hash 0xCBA.
2. A hacked game developer publishes a malicious update to its game (with millions of users), this update includes a shader (same hash 0xCBA) that would display "give me 10 BTC" message all over all windows instead of a shadow as the WM developer intended.
3. The user plays the updated game, the shader gets compiled.
4. The user updates their WM.
5. The user reboots, and finally sees something evil on their screen.

[geearf]

Yes, but it still should be a cryptographically secure hash, just not necessarily very strong or time-tested. The fastest non-cryptographically-secure hash algo is xxHash which is 10x faster than BLAKE3, but it's very easy to get collisions. WMs/compositors use shaders (probably) to display a window's shadow, let's say this shader's hash is 0xABC. Now imagine:
1. An upcoming version of the WM switches to a new shadow shader whose hash 0xCBA.
2. A hacked game developer publishes a malicious update to its game (with millions of users), this update includes a shader (same hash 0xCBA) that would display "give me 10 BTC" message all over all windows instead of a shadow as the WM developer intended.
3. The user plays the updated game, the shader gets compiled.
4. The user updates their WM.
5. The user reboots, and finally sees something evil on their screen.

I was actually wondering why they didn't pick xx so thanks for that!
In that case do you think it would be bad to use it for an FS checksum?

[pkunk]

I was actually wondering why they didn't pick xx so thanks for that!
In that case do you think it would be bad to use it for an FS checksum?

Generally you want non-cryptografic hash for checksum because of speed and possible added redundancy. And cryptographic hash for indexing untrusted data. See https://lwn.net/Articles/529077/

[markg85]

When digging to a couple of commits, this caught my attention: https://gitlab.freedesktop.org/mesa/...requests/22387

Instead of trying to find and integrate SHA-1 assembly from a variety of places, why not replace it with something more modern?

This MR adds a copy of blake3 hashing library into Mesa. BLAKE3 was chosen because it's fast, and there is an actively maintained official implementation, which is used by ccache and llvm.

and:

main (e84cf80f): 5824ms

This MR: 2393ms (-59%)

Just thought that's be handy to post here

[stupotace]

When digging to a couple of commits, this caught my attention: https://gitlab.freedesktop.org/mesa/...requests/22387


Just thought that's be handy to post here

More context about what the 59% applies to is helpful. Thanks for the link!

As a preliminary benchmark, I replaced all sha1 calls with BLAKE3 truncated to 160-bits (not in this MR), and ran fossilize-replay with a warm shader cache (cache hits only). I used --loop 20 which also happens to shift the majority of cost to hashing, because this way Fossilize only does the JSON parsing once. Still, it sufficiently demonstrates how faster BLAKE3 hashing is.

main (e84cf80f): 5824ms

This MR: 2393ms (-59%)

[AndyChow]

Nice to see that people start thinking about the properties of hashes instead of just using SHA256 (or MD5) just because it's the first one that comes to mind. Protecting from attacks isn't the point here, so you want a fast hash, not a slow one.

Sha256 is the first that comes to mind because it's accelerated by virtually every cpu out there. Not sure what the logic here is. Blake3 is faster when there are no AES instructions, but that never happens. Even on ARM chip they have AES accel at this point (well, most do).

Shader hashing happens on the cpu, right? I don't get the logic here at all.

[markg85]

Sha256 is the first that comes to mind because it's accelerated by virtually every cpu out there. Not sure what the logic here is. Blake3 is faster when there are no AES instructions, but that never happens. Even on ARM chip they have AES accel at this point (well, most do).

Shader hashing happens on the cpu, right? I don't get the logic here at all.

It's simple really.
blake3 uses another algorithm that is just much more efficient for the CPU and thus beats the SHA-familly even with their CPU instruction optimized versions.