Maybe we shouldn't skip SHA-3
cryptologie.net
cryptologie.net
SHA-2 512/256 is much faster than SHA-3 and supported by more libraries.
The notion that by recommending SHA-2 512/256 you're setting people up to use prefix MAC SHA-2 512 or SHA-2 256 is kind of silly. You could similarly argue that by telling people to use SHA-3, you're risking that they "fall back" to SHA-2. Either way, you're calling for a specific hash.
The reality is that some people like SHA-3 because it's interesting and because its primitives can be used for a variety of non-hashing applications. That's true! Nobody is saying SHA-3 shouldn't exist. They're just saying: there's no good reason to use it in a modern application.
(If you're not clear on length-extension attacks: they're the reason we use HMAC. HMAC-SHA2 isn't vulnerable to length extension attacks; neither is HMAC-SHA1 or HMAC-MD5 --- both of which, fun fact, can't currently be attacked, despite the weakness of their underlying hash. But if you use SHA-2 512/256, you don't have to use HMAC.)
Would another variant of Keccak, not the one that became SHA-3, provide more competitive results? Or does Keccak in general not have compelling properties compared to the alternatives?
The problem is, there's just not a lot of reasons to use it. SHA-2 isn't broken; in fact, there are hash cryptographers who think SHA-2 may never be broken.
It is a dangerous assertion, especially if it makes people build things that aren't future-proof enough to support more than one hash type.
Otherwise, KangarooTwelve is faster than SHA-2[0], which may be an incentive. Although it is very late compared to BLAKE2, and as a result, it is present in much fewer libraries.
[0]: https://twitter.com/KeccakTeam/status/834789451708628995
If you can't rely on raw computational power to force obsolescence, you need either a major paradigm shift (a la quantum computing) or a clever cryptanalytic attack that bypasses the actual difficulty.
Quantum computers do not currently pose a serious threat to SHA-2. Grover's algorithm can offer a quadratic improvement to collision identification, but not an exponential one. That's impressive, but not enough on its own.
That leaves the last category, which is a clever cryptanalytic attack. This is possible, especially with novel mathematics. But it's not reliable for predicting risk.
So really, the only "future-proofing" we can do for SHA-2 is against vague improvements in the underlying math. In any practical sense of the term, it's not a dangerous assertion to claim SHA-2 may never be broken. As Thomas said, research will certainly continue on cryptographic hash functions regardless; moreover, "improvements in math might happen" is not reliable enough to calibrate forced obsolescence or regular updates to hash standards in the future.
Given the foregoing, some researchers choose to act as though SHA-2 will never be broken, because there's no meaningful way to assert that it probably will or to coordinate when or how it will, and because there are so many more productive areas of research to focus on where the current algorithms absolutely will be broken in the future. You can't productively work on future-proofing something any further once the space of coherent threats has been reduced to, "this might happen in the future somehow" without real specifics.
(This is, I think, JP Aumasson's argument.)
Definitely: it doesn't make much sense to move away from a hash that doesn't seem in danger of being broken, just because it is old — a clever new attack can be found for a newer hash, too (or for both).
Besides, usually there's a lapse of time between finding a major weakness and a practical collision.
On the other hand, when picking a hash… well, having a faster and less error-prone hash is nice.
Overall, I feel lucky to have such great choice.
It is in theory possible that a new technique could break SHA-3 and not SHA-2, but the opposite seems far more plausible.
For example, DES held out for about 20-25 years or so, with some cracks showing earlier than that. AES is now ~20 years old, and we don't have a discovered a single crack in it. SHA-2 is a similar vintage and no significant issues have been discovered. The previous generation of hashes only resisted ten years or so until the first real issues showed up.
Regarding another variant: yes, you can get more compelling results, and the authors of Keccak themselves have also proposed another function based on Keccak called KangarooTwelve[1]. K12 has much better performance and parallelism than SHA-3 does in software. (Of course, if you don't need most of Keccak's general properties, you're probably fine with other hashes anyway.)
Now if I would personally work on a project I would think of SHAKE first, and I would seriously consider KangarooTwelve (especially for tree hashing, which makes K12 better than any other hash to hash big files).
I really don't understand the argument that people can safely type "SHA-3" but not "SHA-2 512/256". Would it help if we renamed SHA-2 512/256? We could call it SHA2.5: better than SHA-2, not quite SHA-3. Or SHA5: better than both!
Renaming SHA-2 512/256 sounds a lot easier than getting new hash code implemented in every library.
> without requiring the adoption of new hashing code
I should have made myself clearer: I recommend SHA-3 for new projects. I wouldn't recommend people to switch to SHA-3 if they are already using SHA-2.
Anyway that was my point! Just wanted to write it up :)
It's a published FIPS.
Edit: this comment got 7 upvotes before the editing timeout, so I'm not the only one either.
Surely SHA384 is no slower than SHA512/256 since both need to compute a SHA512 hash before truncation?
> and the number of truncated bits, which is the factor mitigating length extension.
Is there really a meaningful difference in mitigation between truncating 25% and 50% of the original hash?
The name is a serious problem. If you work with this stuff regularly enough to know the difference it might seem obvious, but that's not the case for many would-be implementors and bug-fixers.
The name makes communication problematic enough to not recommend usage.
Seriously, if you care that much about the name, rename it; Even systems that use SHA-3 use HMAC (SHA-2 is perfectly safe, no matter what variant you use, in HMAC). Name the prefix MAC (an idiosyncratic construction) that uses SHA-2 something specific.
The SHA-3 advocates have already conceded this point. They don't call SHA-3's prefix MAC "prefix MAC"; they call it KMAC, which is a specific construction.
So all you have to accept is that there's SHA2MAC --- which, like KMAC, is not simply the hash function, but instead the optimal MAC construction built on the hash --- and then define it as the prefix MAC built from SHA2-512/256.
It's hard enough to follow this topic for many people that want to keep updated. The fact that you have to spell out "please use SHA2-512/256, but not SHA2-512 and not SHA2-256" is mind-numbing.
"It's really SHA2-512/t where t has a value of 256 and that means that it's SHA2-512 truncated down which alleviates length extension attacks." Just typing that sentence gave me a headache and I have no real idea if it's right or not.
Adoption would be easy if communication was easy. As it is, it's a PITA to even have a short conversation on the subject.
That's what I'm trying to articulate. Hashes aren't MACs.
This is partially my fault, for playing loose with SHA-2 512/256 vs "The Prefix MAC built on SHA-2 512/256".
Again: I think we should just call that SHA-2-MAC. It's an unambiguous name that couldn't mean anything other than "truncated SHA-2 512 used as a prefix MAC".
Also, just looking at a reasonably recent version of Python 3.6: (dir(hashlib))
"'blake2b', 'blake2s', 'md5', 'new', 'pbkdf2_hmac', 'sha1', 'sha224', 'sha256', 'sha384', 'sha3_224', 'sha3_256', 'sha3_384', 'sha3_512', 'sha512', 'shake_128', 'shake_256']"
So - no reference to sha2_512/256 as a unique algorithm there either. Nothing in https://docs.python.org/3/library/hashlib.html either. So, the whole SHA-2 512/256 thing isn't that straightforward.
With that said, I would hope that anyone who was doing this for a real purpose would at least take 10 minutes to read the wikipedia page. And lots of useful information on crypto.stackexchange.com as well.
"In March 2012, the standard was updated in FIPS PUB 180-4, adding the hash functions SHA-512/224 and SHA-512/256, and describing a method for generating initial values for truncated versions of SHA-512. Additionally, a restriction on padding the input data prior to hash calculation was removed, allowing hash data to be calculated simultaneously with content generation, such as a real-time video or audio feed. Padding the final data block must still occur prior to hash output"
This sounds like more than just truncating SHA-2 512. Also, the various articles on stackexchange state that there is no way to truncate SHA-2 512 and conform to SHA-2 512/256.
Finally, https://eprint.iacr.org/2010/548.pdf sounds like they really wanted you to be able to distinguish SHA-512/256 from a truncated SHA-512:
"In order for users to be able to distinguish between a SHA-512 digest which has been truncated and a SHA512/256 digest, we also offer new initialization constants, analogous to those used in SHA-384."
None of this is security-relevant, including the IVs. You would have morally the same system if you just chopped 32 bytes off a SHA-512 hash.
I am aware that other modern hash functions like BLAKE2 (and maybe BLAKE/Skein?) support tree hashing, is there any reason why K12 would be better for that?
Do you agreee with that advice?
(Also, xxhash)
Pardon my frustration, but: Jesus Christ, people. Why do we need to sell everyone on the fancy new thing in a hypothetical situation where it is defined not to matter?
I guess it's not entirely equivalent but I'm still sorta reeling from a "why should we introduce [something like bcrypt] for password storage when we already have these other perfectly fine hash functions in use in our codebase" convo at a previous project.
The thing is that even though security might not be an issue in your application, you might discover in the future that it is. For example python dict relies on hash table to store key-values, doesn't seem like something that would be concerning (you're not encrypting or keeping anything secret), but python applications then started being targeted by collision attacks. The attacks' goal was to insert many elements that hashed to the same value, which caused DoS. Their solution though was to use salt value which is different on every run.
The point is that something that might not seem like a security issue might turn out to be later on. It doesn't necessary mean we should always strong cryptographic hash functions (in many cases they might be too expensive) but at least write code in such way that we can easily replace it with something else if we need to.
If you're using it as part of an auth mechanism then sure, don't use it. If you're using it for anything else (checksum, verification etc.) where you don't expect any adversarial action then it's fine.
I've seen many claims from good engineers of the sort "this design is ok because it's not for security" when it absolutely was.
A simpler rule wins, even if it costs a bit now. It'll save someone in a bigger way later.
...there are even faster implementations of BLAKE2 around. E.g. Samuel Neves' AVX2 implementation is almost 40 % faster on Haswell. libsodium ships a different AVX2 implementation (considered experimental AFAIK by its original author, so I find that a bit strange).
"if it is on the checklist, then.. it is on the checklist"
Truncating from 512 bits to 256 bits hides 256 bits of the state from the attacker, so in order to use a length extension attack they would need to "guess" those bits.
sha-2 256t or something would be much clearer. As the article notes, changing it now is not trivial, but given the potential for confusion is something that might be worth doing, none the less.
[1] Or else a conspiracy theory. Not that conspiracy theories are never true; I'd be surprised but not shocked if this one were.
[1] The drafting of which is based on bikeshedding/NIH internal politics.
I think the point the OP makes is that a suggestion like "use SHA-3" is a simple, succinct, and not-unacceptable answer to a question like "what cryptographic hash function should I use?", giving a safe albeit conservative, but instantly graspable answer without having to go into additional detail -- other than the obligatory mention to that general-purpose hash functions aren't by themselves appropriate for key derivation ("password hashing").
The alternative view -- that SHA-512/256 (which suffers from its naming), or the longer-but-less-truncated-state SHA-384 is faster, more studied, more widely supported -- is a more nuanced recommendation, but then you have to explain why you don't mean SHA-256 or SHA-512. The innovation of libraries like nacl and libsodium was to the user of crypto from having to be a crypto expert themselves, and once you have to explain which of the SHA-2 hashes specified in FIPS PUB 180-4 you should and shouldn't use, we're not really any better than in the footgun days.
A much more general argument is that most people should be using cryptographic frameworks (e.g., which implement TLS), and Adam Langley's thoughts about whether or not SHA-2 should be skipped should be aimed at people who are creating those cryptographic frameworks.
But if we are giving advice to random application programmers, they shouldn't be trying to pick cryptographic algorithms to begin with, and the question of whether you should be using SHA-2(key || data) is the sort of thing where the Zen master would be hitting the student with a cluestick for asking the wrong question to begin with.
I mostly use hashes as part of signing/verifying small messages, say an 80 byte JWT, a Blockchain transaction, a certificate, an TLS/SSH packet, etc. Besides hashing large files (which I rarely do), I don't see where I would reach asymptotic performance, or even use tree-hashing.
SHA3's block size is 200 bytes, KangarooTwelve's is, if I'm not mistaken, 8192? I'm more worried about not even filling the first block :)
The number I used is definitely not the best one. But I believe it applies to you as you're not hashing a lot of bytes so you will not see much of a speed difference between hash functions.
> SHA3's block size is 200 bytes, KangarooTwelve's is, if I'm not mistaken, 8192? I'm more worried about not even filling the first block :)
They have the same block size :o rather 8192 is the threshold before you start using the tree hashing mode. This way if you want to hash small messages you do not get a penalty from using KangarooTwelve! It's all good :)
That's a strawman. It's NOT useful.
EDIT: What's the one downvote for? Just curious!