Hacker News Reader

Comment by JoachimS | Hacker News Reader

Regarding MD5. I advice people from using MD5 in new designs even if it is for HMAC-MD5. Not because that construct is broken (which it isn't), but because we add yet another dependency on an algorithm we in general consider too weak to use.

Do you agreee with that advice?

View on HN

BLAKE2 is faster than MD5. I think that means there's no longer any reason to use MD5 in new applications, if you have any choice.

While the portable, pure C implementation of BLAKE2 that is used almost exclusively by libraries (e.g. Python, OpenSSL) is already faster than MD5...

...there are even faster implementations of BLAKE2 around. E.g. Samuel Neves' AVX2 implementation is almost 40 % faster on Haswell. libsodium ships a different AVX2 implementation (considered experimental AFAIK by its original author, so I find that a bit strange).

For this kinds of thing, I am thinking that it might be worth it to do a "web browser" mode which uses 128-bit vectors and a "web server" mode which use 256-bit.

Yes-ish. I don't sweat HMAC-MD5, nor HMAC-SHA-1. What really sucks though is when you can't use them (even though for interop reasons you must) because some stupid auditor (typically software) goes "oh no! MD5 found!! alert! alert!".

Heh heard that one, and the always very convincing answer in a discussion to do or not do certain things...

"if it is on the checklist, then.. it is on the checklist"

There is lots of junk on those checklists. Like "must implement n-strikes you're locked", which is idiotic in many contexts.

For us who don't know better, could you explain why MD5 or SHA1 are ok in an HMAC context?

Because a collision in MD5 doesn't give you a second preimage of an MD5 hash.

Definitely don't use MD5!

What if the use case is not security sensitive, I just want to implement a distributed hash ring for example?

Make sure it can easily be replaced if needed though. I also use it in contexts where I need some consistent "randomization" (like your hash ring).

The thing is that even though security might not be an issue in your application, you might discover in the future that it is. For example python dict relies on hash table to store key-values, doesn't seem like something that would be concerning (you're not encrypting or keeping anything secret), but python applications then started being targeted by collision attacks. The attacks' goal was to insert many elements that hashed to the same value, which caused DoS. Their solution though was to use salt value which is different on every run.

The point is that something that might not seem like a security issue might turn out to be later on. It doesn't necessary mean we should always strong cryptographic hash functions (in many cases they might be too expensive) but at least write code in such way that we can easily replace it with something else if we need to.

BLAKE2 is faster and has features you might find useful in that sort of scenario. And doesn't forfeit security, should you decide to use this in a an unexpectedly security-sensitive application.

If the use case isn't security critical, and you know for a fact it won't become security critical later, and MD5 is convenient for whatever reason, then just use MD5.

Pardon my frustration, but: Jesus Christ, people. Why do we need to sell everyone on the fancy new thing in a hypothetical situation where it is defined not to matter?

Because now people are carrying an MD5 implementation around with them, and when the next vaguely-hash-shaped problem comes up, they're gonna reach for that without really evaluating, b/c they've used it before so and it's at the top of their mental cache and it's even more convenient now. :/

I guess it's not entirely equivalent but I'm still sorta reeling from a "why should we introduce [something like bcrypt] for password storage when we already have these other perfectly fine hash functions in use in our codebase" convo at a previous project.

No No No. Then you add another dependency on MD5. And MD5 is a very slow, complex hash function. Use MurMur, City or any of a number of really fast, non secure hash functions instead.

A situation where it is defined not to matter is unrealistic. Things you don't think are "security critical" have a tendency to unexpectedly become so; at least that's my impression. Why not use a more secure algorithm?

Exactly this. I dont like when people try to sell new things even though my use case is perfectly fine with the older solution. I understand the security implications, yet there more to this topic than just security.

Then you are using a _very_ slow hash function for which there are many algorithms better than MD5.

If the application isn't security sensitive why not a faster non-cryptographic hash like farmhash?

Maybe because it is less popular, this is first time I hear about it. Probably it's also not as widely available.

I agree. There is also siphash that we use.

These are often not packaged, so less available. Though for some applications you wouldn't want a library anyway.

(Also, xxhash)

I totally hear you. ;-)

It depends what you're using it for, no?

If you're using it as part of an auth mechanism then sure, don't use it. If you're using it for anything else (checksum, verification etc.) where you don't expect any adversarial action then it's fine.

You should not use md5 for anything. It's completely broken and that's just asking for trouble if someone swaps out whatever your checksums are checking. It's worth the security to swallow the extra computational power. Though, if you're hosting something no one cares about, it's still worth the security to keep your small audience safe from the random malicious person you thought you'd never see.

I'd much rather have it as a well known meme that "md5 is broken, don't use it. Period." than a list of caveats like "ok for non security" or "ok with hmac".

I've seen many claims from good engineers of the sort "this design is ok because it's not for security" when it absolutely was.

A simpler rule wins, even if it costs a bit now. It'll save someone in a bigger way later.

Good points.

I think the point they were making was that since we know of weaknesses in certain situations we should avoid using it even in the safe situations so that we can eventually drop it from crypto libraries all together. There are alternatives that we can all move to.

Let's say I make a hash of an asset and put it into a very long-time archive (like 100 years or whatever). What checksum algorithm should I use to be sure it will be readily available in software after 100 years? And to make the choice more realistic, let's assume that the algorithm must be sufficiently fast today (like md5 is).

The answer is that you don't want a hash, you want a solution that is ArchiSig or TR-ESOR (or something similar) compliant.

So what's the right algorithm to use instead of md5 if I just want a fast checksum that's widely supported?

BLAKE2b or BLAKE2bp should be a good choice for that. Or just CRC if you care only about unintentional changes.

SHA256

If SHA-2, do use SHA-512, since we are living in a 64 bit world.

SHA-512/256

Yeah, that's a good argument to move to a stronger hash function for HMAC.

Reply on news.ycombinator.com