BLAKE3 Cryptographic Hash Implementation Preparing For v1.0 Release

[Palu Macil]

Also, what SHA in hardware? The broken SHA1 that shouldn't be used anymore or SHA256 which is susceptible to length extension attacks, which limits its use or requires more complex constructions (with multiple calls that reduce speed)?

You use SHA512 because it's faster than SHA256 anyway (at least in 64-bit systems).

A length extension attack is related to using a hash in a MAC. I don't believe it can cause collisions in the hashes. The problem is with authenticated MACs. The answer is to use a well tested, standard algorithm provided by your standard library like an HMAC. The HMAC is available in virtually all standard libraries and very difficult to get wrong. Ironically, if you use BLAKE instead for an authenticated MAC, you should still be using it within HMAC, which means your "extra work" point isn't true.

BLAKE2 is similar in terms of security to SHA3. So if you can use both SHA512 and BLAKE in an HMAC, why do you pick SHA512? Well, you do so to be able to sell your product to anyone. Nobody in the US is buying software outside the mainstream best-vetted cryptography. In many industries they can't for regulatory or certification reasons.

[Davonious]

OK - talking about the NSA/NIST subverting the security industry may work on an internet forum, but I assume you wouldn't be bringing that up in meetings with your customers. You'll be saying "our product follows FIPS recommendations"

True, true. We recently dropped a vendor due to their use of non-FIPS certified algorithms. The product in and of itself was fine, but we'd formally raised the issue of non-FIPS algorithms before. After a year (or more) of "we're working on it" and no timeline for implementation, at license renewal we went with a competitor.

Sad thing is that the crypto features seemed secondary/tertiary (at best) to the core functionality of the product.

[ChamPro]

I think this discussion is missing something important. Maybe Blake3 is also really useful in the non-security space? I can think of all kinds of use in the storage space. skeevy420 mentioned it earlier for ZFS. I can see this being incredibly useful for metadata analytics; applications that scan an entire file space, hashing every file, and keeping track of when they change in a NoSQL database. The speed improvement over MD5 or SHA1 is significant when you're looking at over a billion files.

[microcode]

Sure it's a little faster

Dude, it's almost ten times faster.

[ddriver]

There is something important missing from the picture - actual cpu load. If this algorithms is just better at saturating the cpu pipeline, this will inevitably come at the expense of other running processes. It will be faster to hash, but it will drag other workloads lower, so that massive throughput is unlikely to translate to huge real world performance improvement, much like the vast difference between synthetic and real world performance for SATA vs PCIE4 ssd.

[xnor]

You use SHA512 because it's faster than SHA256 anyway (at least in 64-bit systems).

What systems? On a Zen2 SHA2-256 is over 2x faster on long messages and this increases to like 4x for very short messages,

A length extension attack is related to using a hash in a MAC. I don't believe it can cause collisions in the hashes. The problem is with authenticated MACs. The answer is to use a well tested, standard algorithm provided by your standard library like an HMAC. The HMAC is available in virtually all standard libraries and very difficult to get wrong. Ironically, if you use BLAKE instead for an authenticated MAC, you should still be using it within HMAC, which means your "extra work" point isn't true.

Any kind of authentication where these NIST-approved secure hash functions are used can be vulnerable to this problem. This includes APIs that use such hashes as authentication tags, web services, authenticated encryption ...
HMACs are not provably secure unlike other, proper constructions that were designed for the task instead of working around limitations of hash functions that are vulnerable to length extension.
This has nothing to do with collisions per se, but when you use another hack, namely truncation, to gain resistance to length extension you also weaken collision resistance.

BLAKE2 and 3 don't have this problem because they add a finalization step from which you cannot recover the hash function's state.

HMAC requires you to call the hash function twice, sequentially. A keyed hash function that is resistant to length extension does not require such a construction. BLAKE3 further optimized this in not even requiring you to hash the key because it uses they key as part of its IV so to speak.

BLAKE2 is similar in terms of security to SHA3. So if you can use both SHA512 and BLAKE in an HMAC, why do you pick SHA512? Well, you do so to be able to sell your product to anyone. Nobody in the US is buying software outside the mainstream best-vetted cryptography. In many industries they can't for regulatory or certification reasons.

Similar in security but much, much faster. As for selling products I have to tell you that I'm not a salesman.

[stormcrow]

NIST, I assume, isn't subverting security but the NSA is. Anyway, the point was that you don't need a tinfoil hat to have concerns when the NSA is mentioned with secure standards in the same sentence since we have evidence.


But BLAKE using AVX instructions is at the same speed or faster.
Also, what SHA in hardware? The broken SHA1 that shouldn't be used anymore or SHA256 which is susceptible to length extension attacks, which limits its use or requires more complex constructions (with multiple calls that reduce speed)?

Evidence of particular cases yes. Cryptography is not an all or nothing field. Throwing the baby out with the bathwater is insane. Unless you have direct evidence of the NSA undermining a particular implementation it's a red herring argument. For one thing, Intel (so is AMD for that matter) is perfectly inept enough to break its own CPU instructions without help from anyone else. For another, the most wide spread Linux kernel security system, seLinux, was invented by the NSA and has regularly withstood analysis. The NSA isn't a monolithic entity. Parts are dedicated to offensive operations, and parts are dedicated to defensive operations.

[xnor]

Evidence of particular cases yes. Cryptography is not an all or nothing field.

So when something as simple as a random number generator has a backdoor everything based on it is just somewhat less secure? No, it is an all or nothing field, because that single backdoor allows you to break any encryption, any authentication, any signature algorithm that makes use of it.
And the NSA is by far not the only entity of which we have evidence for deliberately undermining information security. Not only to spy on foreign countries but also their own people.

Throwing the baby out with the bathwater is insane. Unless you have direct evidence of the NSA undermining a particular implementation it's a red herring argument.

Nah, it's the only reasonable position to take. You don't assume generally good intentions, especially from an organization that specializes in espionage, communication interception, decryption ... That would be quite gullible and naive.
Instead you should require evidence and mathematical proofs about the security, and for everything they had their hand in you should require even stronger evidence and more proofs and should be more skeptical, not less.

The NSA isn't a monolithic entity. Parts are dedicated to offensive operations, and parts are dedicated to defensive operations.

NSA doesn't publish the algorithms used for high security defensive operations.
The public algorithms are used for short-term storage (which can be securely erased) and interoperability.

[arQon]

For one thing, Intel (so is AMD for that matter) is perfectly inept enough to break its own CPU instructions without help from anyone else.

If only there was some way to test the outputs of those instructions for correctness. Maybe something like a function that turns an arbitrary-length stream of data into a consistent checksum of some kind, with enough result bits that the chances of two different streams producing the same result is infinitesimally small. I'm sure I've seen something like that mentioned a few times in the past.

Not sure what we'd call it though. "Secure checksum" is a bit of a mouthful. It's kind of like a hash though, so we could just call it that for a while until we come up with a better name...

[Palu Macil]

I think this discussion is missing something important. Maybe Blake3 is also really useful in the non-security space?

If you're going to use it for storage or something, you wind up competing with far faster things like xxhash, farmer hash, murmurHash, etc.

EDIT: There is something to be said about the fact that sometimes you think something has no security implications and then later you realize it does.