Hacker News Reader

Comment by gruseom | Hacker News Reader

I have a dumb question about this. It was a smart question when someone asked me it the other day, but it's a dumb question now because I feel like I should know the answer and I don't. Why can't you get away with using some trivial but obscure modification of one of the standard fast hashing algorithms? It will be just as vulnerable as the standard algorithm, of course, once you know what it is. But now the attacker has to figure out which algorithm you modified and how you modified it. How do they do that?

I get that this is a bad idea that won't work, blah blah security by obscurity and so on. But when I was asked why it doesn't work, I was unable to give a very satisfying answer.

View on HN

These things are really hard: http://en.wikipedia.org/wiki/National_Security_Agency#SHA. Here we see the NSA suggesting a modification to SHA-0 to make SHA-1. For years, incredibly smart mathematicians didn't see what the NSA saw. So, I'd say there's a decent enough chance that you'd weaken the security. I don't think many people on here would have the crypto chops to make their on security algorithms - people like cpervica are the exception and even they want to publish their algorithms.

I mean, there is a certain logic to what you're suggesting. Let's say you're cpervica and you know what you're doing. You make mistakes like any human being, but you know crypto. So, some cracker gets access to your database. OK, they're probably not smart enough to find a weakness in what you've done and might not even be able to figure it out. But there might be a weakness there.

Plus, you have to think: what if someone gets my code and my database? Then they have the modification you made. If the modification doesn't require more computation, then it's just unknown without the source code. So, with the source code, we're back to the trivial to cracking case.

The thing is: there are solutions out there to handle this by making the calculations take longer. PBKDF2, bcrypt, and scrypt all exist. They deal with this specific problem in a way that even if someone gets your hashes, salts, and code, you're less vulnerable.

tl;dr: with the obscure case, you're not gaining protection if they get your code along with your database.

Well if they don't have your source code, why modify the algorithm, you can just use sha1(password + user_salt + site_secret) -- that site_secret just made the sha1 unique to your site. Of course, if they have your source code, then it doesn't matter: they would have your 'site_secret' or your modified algorithm. Better would be not storing your site_secret in a accessible way (not on disk).

Edit: See udk1 below -- he's right, sha1 is an outdated algorithm for this purpose. Poor example choice on my part.

You shouldn't use SHA: it's not designed for hashing passwords. Checkout bcrypt / scrypt and read: http://crackstation.net/hashing-security.htm

That doesn't mean an additional secret stored on the server is a bad idea either way.

I've heard it referred to as a pepper (to go along with salt) and is in the application code.

bcrypt(password, salt, pepper) => hash. So even if they grab the db with the salt, the effective password to crack turns into 'password3jkl453jklgfuja9oph4mn" instead of just 'password'. Impossible.

And if they do get your site's code, you're back to a strong hashing algo and a salt.

It would work, but a) you might break the algorithm in ways that make collisions b) if your code is stolen in a breach, it's useless.

Most people should use bcrypt with enough rounds to make it difficult to brute force. scrypt is also an option, though newer (good / bad: http://security.stackexchange.com/questions/26245/is-bcrypt-...).

The attacker has pairs of hashes and known passwords (for instance, for their own accounts) and only has to figure out what the mapping is for those; the answer then extends trivially to the rest of the passwords.

That's the "theoretical" answer. The real answer is, popping someone's database virtually always --- you know what, let's just assume always for now --- gives you a remote shell, and from the the actual code.

Ah, that theoretical answer does fill in one missing link for me. I kept wondering how you could detect even a trivial modification of a standard hashing algorithm (say, reversing the hash) if all you had to look at was its outputs. It seemed like a massive search problem through the space of all possible modifications, though no doubt there exist sophisticated techniques to apply. But if you have some input/output pairs, the problem seems entirely more tractable. Even I can begin to imagine how one might tackle that.

The point you and others make that if the database is compromised then the code almost certainly is too, is clearly the best answer though. I am now prepared for the next time I encounter my interlocutor at extended family dinners!

In my house we have a rule that we don't discuss password hashing at the dinner table

Cryptographically, you can address the problem you've set out for yourself simply by hashing a 128 bit random number along with the password, and keeping that number a secret. If attackers can get your code, it doesn't matter how you obscure your hash, because they'll have the algorithm. But through trial and error, an attacker might figure out how you tweaked an algorithm; all the atoms in the solar system (or something like that, I can never remember) could be computers trialing and erroring against a 128 bit random number and they'd never figure it out.

128 bits is a big number/keyspace but the solar system is no lightweight, either.

    2^128 = 3.4028E38 
    solar mass = 1.9884E30 kg or about 9E56 atoms.

You don't need to get down to atoms, you could be crunching 128 bit keys on a single AWS dyson1-medium instance.

256 bits is the one where the hosting costs start getting not merely planetary but intergalactic.

    2^256 = 1.1579E77
    atoms in Milky Way = 2.9E76 atoms

You're presuming that "obscure or trivial modification" to a hashing algorithm would be just as secure (or just as vulnerable) as the original. That is no guarantee. It's entirely possible that you might fundamentally break the algorithm and make it much much less secure.

This secure stuff is hard. Remember when debian made a "simple fix" to open ssh and made all the ssh keys for 2 years brute forcible? Do you think you can do better? It's wiser to leave this to the experts

Debian made a dumb software coding mistake that broke entropy input into OpenSSL's RNG. This is a part of the code that's very hard to test for correctness because by its nature, it must behave unpredictably.

This could have (and often does) happen to any software. It's not a good example of someone attempting to design their own or modify an existing crypto primitive algorithm.

It's a good example of how someone can look at an algorithm/software and not realise what bits are critically important. "Oh this bit I can change without anything bad happening". It shows there are unknown unknowns in security.

Well the Debian maintainer did realize this was important and emailed the OpenSSL developers. But due to a tragi-comedic miscommunication the breaking change got committed anyway.

If they had access to the database, it's not unreasonable that they had access to the code

I think perhaps the satisfying answer you are looking for is: http://en.wikipedia.org/wiki/Kerckhoffs%27_principle

Yes, it's a great idea.

It's called "salting" the password. Everyone does it. It works really well.

Of course, if they can steal your salt (from your source, or a config file) then you lose.

What's the OP is talking about is commonly called a 'pepper'. Some fixed secret in your application code that impacts the generated hash.

Since salts are random, and unique per password, you shouldn't be storing them in a config file.

Reply on news.ycombinator.com