Please stop hashing passwords
blog.tjll.net
blog.tjll.net
> Stop associating “hashed” with “secure” when it comes to passwords. If you’re storing user credentials MD5-ed without salt, you’ve put an thumb tack in front of a steamroller - just a minor annoyance that won’t offer you any real security. You have to do it right or hashing means nothing.
> Start relying on key derivation functions for password storage.[1] Are they perfect? Will they be the best practice in five years? Probably not, but they’re your best option when rolling your own user credential storage.
> Don’t sacrifice security for performance. A slightly slower malloc in OpenSSL would have been much preferred, in my opinion, than the disaster that is Heartbleed.
- [1] There’s a fair bit of dispute over the ideal KDF for storing passwords, but generally speaking, any of the three covered in this topic are popularly accepted options.
It's not just about finding and fixing bugs, but how to improve the whole process.
But let's go for a part 3: please try not to store user credentials at all, if you can avoid it! It's not a great idea for every random Web site to create its own notion of identity for every user -- which requires a means of proving that identity, usually a password, and a way for that means to get compromised.
When it's feasible, better would be to outsource the job of verifying user credentials to some third party and only accept online cryptographic proofs of identity, whether that's via:
* an SSH client keypair (e.g. git push to GitHub)
* requiring requests be signed with a PGP keypair (e.g. uploading to Debian/Ubuntu)
* Mozilla Persona/BrowserID
* a TLS client certificate
* Facebook Connect
* Google Accounts
* OpenID
There's another benefit here -- no matter how many times you iterate a KDF, your website is still going to be vulnerable to an online compromise where the badguy just grabs passwords as users log in. (I understand systems like Meteor are nudging sites into using SRP-in-JavaScript, which is great, but in an online compromise that can just change.)
And while it may be good to use bcrypt vs. SHA-1, after all it's the users whose interests are ultimately at stake if the password DB is revealed, and yet the users generally have no way of knowing (cryptographically) that any given site really is using bcrypt. That suggests that the security dust is being applied in the wrong place -- the party that actually cares should be able to verify that the right thing is happening.
It would have been great if Mozilla had had more resources to stick with Persona (and of course if Facebook and Google could have afforded to endorse it).
Unfortunately even sites that are founded by ex-Facebook employees (like Quora) don't actually trust Facebook Connect enough to rely on it -- Quora uses FB Connect to let you "Sign Up with Facebook" but then requires you to establish your own Quora password just in case Facebook screws them over someday. (Then they store the password via unknown means...)
Maybe it's just me. But if many others on HN feel the same way, then that could be relevant for a new startup who wants to gain traction among a core group of initial users. E.g. I never would have used Dropbox or Airbnb if their only option to login was Facebook.
Do you not want the identity provider (e.g. Google) to know every place you log in? (This was something Persona solved, alas...)
Do you not want to be reliant on a megacompany like Google whose spam filters might someday hit a false positive, causing Google to ban your account, and there's nobody to call and you're locked out of everywhere?
Or is it about separation, i.e. you don't want any single notion of your identity to have too much power if compromised (and you're careful to use unrelated credentials, e.g. distinct passwords, on every website)?
Or something else?
Are you ok with the way that GitHub and Ubuntu outsource the storing of credentials to authenticate a "push" by checking against a public key, for which only the user holds the private key? What if more services worked this way?
Do you not want to be reliant on a megacompany like Google whose spam filters might someday hit a false positive, causing Google to ban your account, and there's nobody to call and you're locked out of everywhere?
Or is it about separation, i.e. you don't want any single notion of your identity to have too much power if compromised (and you're careful to use unrelated credentials, e.g. distinct passwords, on every website)?
Indeed, those are some excellent reasons to avoid any centralized login system. :) Most people won't care, but early adopters might. Startups don't need to care about early adopters after the 'early' stage, but the early stage is critical, so it's just something to keep in mind.
Are you ok with the way that GitHub and Ubuntu outsource the storing of credentials to authenticate a "push" by checking against a public key, for which only the user holds the private key? What if more services worked this way?
That'd be lovely. Unfortunately the key management problem hasn't really been solved: there's no way to make it easy for average users to create a key and use it on a bunch of different devices. "What you know" (a password) is still way more convenient than "what you have" (a keyfile), unfortunately.
I don't think there's any way to solve that without using a third party to sync keys across your devices. Something like that might be able to be done securely, but it'd require a lot of thought and care. (Ultimately we have to trust the service provider with our credentials anyway, so trusting them to sync keys doesn't seem like too far of a stretch.)
You've just listed four good reasons to reject centralisation of credentials with those organisations, and any one of them would be sufficient disincentive in my case.
I can think of at least one more good reason: If I'm dealing with an organisation, particularly one I'm paying for their services, if I have unique credentials with them, and if anything goes wrong on their side, then it's going to be their responsibility to fix it and make good any damage. If I'm dealing with them and using a third party for credentials, and that third party gets compromised in some way, then it seems likely it will end up being my responsibility to sort everything out and pay any related costs.
No, I do not trust megacorps to keep my credentials safe.
I do not wish to use the same credentials for every site, for several reasons: in particular, worse consequences of account compromise; worse consequences of terms of service changes.
No, I definitely do not wish megacorp identity provider to be able to link information from my accounts across different websites.
No, I definitely do not want to be reliant on a megacorp. Amongst other reasons, I do not believe that a single entity can provide for all cases and competition in solutions is a good thing.
I don't want my account at random website X to be dependent on external services. Random website X will be slower and less reliable, and I will be less able to access support.
I do not want any single notion of my identity to have too much power if compromised.
I would rather accept the risk that random website Y is storing my password in plaintext rather than accept the risk of using a megacorp (or other external service).
I have a couple of problems with identity providers. One is that they are, by necessity, a single point of failure. And because of that, they are more jucy targets. When someone exploits the provider, all of a sudden they can access any website that I have used that provider for. When the provider goes under or decides to stop operating the service, I'm probably SOL. If the service has an outage, I cannot use any website using it.
I also have issues with companies selling my personal info. At this point, yes. I do put more trust in a random website keeping my credentials safe for that website than Google keeping my credentials safe for every website that Google has in its clutches.
Public/private key crypto can work, but has issues of its own.
Why you list some of the reasons against using those businesses, it's important to remember that some of don't have accounts with those businesses. Never have, and never will. So requiring FB or G accounts to participate ends up being is a rather offensive "we don't serve your kind here" door-slam.
You can offer them as an alternative, but requiring a troublesome 3rd-party like that is only going cause problems.
> do you place greater trust in AirBnB (or random websites X, Y and Z) to keep your credentials safe than you do in Google?
YES. By a wide margin. Some random website I can give a unique email and password, so if they do bad things, the damage is limited. I wildcard all the smail at my domain to the same place, so if I see spam addressed to e.g. "airbnb.domain@mydomain.net' I know who sold the address to spammers and have an easy regex to filter on, if necessary.
With google/etc, I wouldn't have that kind of granularity. Of course, there are all the usual reasons like not wanting google to know every login I make, but that topic is already covered.
> checking against a public key
That would be outstanding. Yes, we have not solved the key-management problem yet, but that's no reason not to try. With a problem this complex, it is going to take a number of attempts anyway before we get it right. Waiting for a "perfect" solution, on the other hand (i.e. "Web-of-Trust is too complicated!") will just leave us with the current mess.
Or are you worried about whatever data you put on the service after you've logged in?
Signing up with a third-party log-in doesn't really do much to protect the data you enter into a site after you've logged in.
For every project I've worked on that had both FB Connect and in-house login, almost no users used FB Connect, no matter how the two options were visually presented. I've seen iOS apps get absolutely HAMMERED in their App Store reviews simply because they don't have any login other than FB and users refuse to use it.
I agree this is a problem, but if I'm a line-level engineer or product guy, my short-term goal isn't to come up with a high-level solution, it's to do what works and won't fuck with my acquisition funnel. Which is currently roll your own.
Users don't trust the app won't do anything else beside authentication, like: posting to your wall, spamming your friends, reading all your information, etc, etc
Sorry for nitpicking, but in my understanding of terminology, non-anonymous authentication always require some sort of credentials and a public key/OpenID/Google account/whatever is one, just as well as an old good username-password pair.
And in case of verifying certificates/signatures, we're not outsourcing jobs to any third party, but doing the verification ourselves. This is important distinction between user-owned keypairs and Persona/Facebook/Google/OpenID - I'm not sure entrusting user identities and authentication to a third party, instead of establishing secure means of authentication, is a wise decision.
Better approaches:
1) Let the client do expensive KDF (e.g. in JavaScript if it's a web app) and send you only the resulting key. You can then store SHA256(key) instead of KDF(password, salt).
2) Use public keys. This requires some fancy infrastructure like a keychains, keybase.io, PGP software etc, but it is the most secure solution. User saves only his public key on the server and signs authentication token when he wants to get in. Server has absolutely no secret material to leak, while the user has complete control over safety of his keys. This is how SSH works. The reason it is not widely used on the web is because we don't yet have nice infrastructure: nice UI for keychains and protocols for one-click sign-in via crypto signatures. We all should work on this to make it better for our kids ;-)
So $hacker has to simply do a rainbow-table attack... no security gained there, except for a vastly bigger key space.
It's relatively trivial to rate-limit access to particular endpoints if you are particularly concerned about DoS attacks, which means you've got a simpler system overall – an simplicity in security is reaaaaaallly worth it!
This tells me that the HN crowd still hasn't got it.
My own attempt at this message: http://www.justgohome.co.uk/blog/2014/02/salting-hashing-and...
Since the GP mentions Django, it's worth remembering in this connection that there are extra security concerns with expensive KDF algorithms. There was a vuln in Django where people could submit 'passwords' of arbitrary size - a 20 character password isn't a problem, but 1MB worth of text being expensively hashed a few times at once could be used as a nice DoS vector.[0] So there's more to the problem than just bunging in PBKDF2/bcrypt and forgetting about it.
[0] https://www.djangoproject.com/weblog/2013/sep/15/security/
It doesn't matter, though. PBKDF2, bcrypt, scrypt: all fine. Just stop using salted, peppered, or lightly-battered hashes.
If so, why say "don't use hash functions and use KDFs" when KDFs are used as hash functions in such cases? Maybe it's better to say something along the "don't use fast hash functions, use slow ones" lines?
But would it be (more) correct if we'd say "key stretching function" instead of KDF?
The problem is that the same word "hashing" is widely used (incl by experts) for 2 different things, normal insecure hashing and cryptographic hashing. As he points out, the 2 have different purposes, ie CRC for large files & git blobs, etc for the first, and cryptographically secure storage of small strings for the 2nd.
KDFs meet all the requirements of cryptographic hashing. Maybe a new term is needed for the process of cryptographic hashing, like "scrambling", or something that suggests secrecy, rather than just generating a unique value.
If the author is reading here, it's probably best to take PBKDF2 out of the graph until you get it properly configured. Otherwise, people may choose sha3 as their password hashing function.
(if you have a place to keep this extra secret where such an attacker can't get it, why not just put the passwords there?)
Using a pepper is another example of security in depth. It doesn't protect against the situation where your attacker has root access on your server, but it increases the work that an attacker has to do (or the luck that the attacker has to have).
If you assume that the password handling code is on a webserver and the database is on a separate server, there is no reason to assume that an attacker getting read access to the DB through an injection attack will be able to access a pepper on the webserver.
Defense in depth is rarely a bad thing. Especially when it doesn't actually harm the end-user experience.
Second, even with SQL injection, keeping a secret key somewhere on the disk is unlikely to be helpful, because SQL injection attacks usually cough up remote code execution, and even more frequently provide direct read access to the filesystem; on many hugely popular frameworks, "arbitrary file read" is itself remote code execution.
Unless there is some extra network vulnerability that lets them then hop to the webserver to read the secret pepper straight out of memory, I can't see why it isn't worthwhile. It may only provide a modicum of extra security, but it also is trivial to implement. Unless there is a cryptographic downside to adding the pepper (possible - outside my area of expertise), I can't see why not?
You're starting make me feel odd for not having the database on the same server as the web service now. I feel we've probably got some crossed wires here...
The argument is that you can put the key into the code base or into the deployment environment. Passwords are user changeable and can't go there. And, just maybe, you only lose one or the other but not both.
(NB: I'm not defending the pepper, but at least one very major software security consulting firm uses this argument. I don't know how much time they spend doing pen tests and getting a feel for when the pepper would have actually saved anyone's ass, and when I pressed I only got a vague mumble about customer confidentiality.)
Salt is not hidden, if you get the passwords, you get the salt
But then you might suspect user X is using a weak password (that you know)
Then you use that to bruteforce the pepper (a defence would be to have a big pepper)
Or you know, if you already got in and got the password hashes it shouldn't be too difficult to get the source code/config info.
All in all, it seems like a lot of work (that could be spent elsewhere) for little gain. I've been burned enough to know that any tweaks to a crypto system to "make it better" sometimes shoot you in the foot instead.
I noticed in Django's admin this line (in the user section, with the default authentication app):
algorithm: pbkdf2_sha256 iterations: 12000 salt: 8c8ttQ****** hash: pN+2tq**************************************
I think it's fine, although some people believe [sb]crypt to be more secure. I think PBKDF2 is actually the default in ASP.NET now.
Edit: Django has support for bcrypt but it's not the default because of third-party dependencies[1].
[0] https://news.ycombinator.com/item?id=7596280
[1] https://docs.djangoproject.com/en/1.7/topics/auth/passwords/
Django chose PBKDF2 of the three primarily because it was easily implemented without pulling in third-party dependencies, which is very helpful for Django's deployment. Given the choice of the three, I think most all Django core developers would say that scrypt or bcrypt would be better but of the three options, PBKDF2 was deemed to be the most acceptable for driving adoption. Before PBKDF2 was added it was using a fast hashing algorithm and had no way to select one so all three were considered improvements and PBKDF2 was considered the 'safest' for adoption, not the optimal one. Note that it is now fairly easy to add bcrypt or scrypt into your django deploy if you choose to.
I'd also add that the parent article, talks about PBKDF2 but I am not sure it is using it correctly. It makes no mention of how many rounds it is configured for and I'm not sure what they mean by 'dumb iterations'. It should be able to make it as slow as scrypt or bcrypt or on a single machine benchmark. The advantage of something like scrypt is that it pulls in complexity on memory as well as CPU.
Edit For anyone interested in the decision making here, this was the summary ticket on Django password hashing and my summary at the time:
https://code.djangoproject.com/ticket/15367
https://groups.google.com/d/msg/django-developers/ko7V2wDVsd...
(That's the passage I was referring to:
> If you're explaining crypto to a reader, and you're at the point where you're discussing KDFs, you'd think there'd be cause to explain that PBKDF2 is actually the weakest of those 3 KDFs, and why.
Also, I'm not arguing the sanity of Django developers' choice of PBKDF2 as default password encryption mechanism—IMO marginally better security wouldn't be worth the increased complexity of starting new project for newcomers.)
It's perfectly okay.
http://stackoverflow.com/questions/4433216/password-encrypti...
It's to prevent a limitation in the password size. (I can't find the page explaining this though) https://docs.djangoproject.com/en/dev/topics/auth/passwords/
You can switch to other hashers, of course, as explained on the page.
Now, is it only my impression or in the benchmark there's a bug and the number of iterations for pbkdf2 is fixed to 1?
That said, don't roll your own. You probably screwed it up somewhere. Just use the bcrypt library call. Or scrypt to let you roll +2 against GPU attacks.
[1] http://www.win.tue.nl/hashclash/rogue-ca/
[2] http://blogs.technet.com/b/srd/archive/2012/06/06/more-infor...
[3] http://link.springer.com/chapter/10.1007%2F978-3-642-01001-9...
[0]: http://vnhacker.blogspot.com/2014/04/fairy-tales-in-password...
A reader of your comment might assume that the article is about some perceived flaw in scrypt itself.
I see it as it being more about incorrect usage or application, rather than a fault with scrypt.
Better: Use one time passwords sent via SMS. Or send a one-time-login URL via email.
If you do have to use a password, just generate a 10 digit numeric code. Sure, some of your customers might complain, but at least you aren't responsible for disclosing people's ebay password when your site gets hacked.
2FA basically ensures security via a second channel, and it's perfectly possible to store passwords in a secure format. I'm not convinced your ideas there are worth the cost.
Why? Almost all websites require email confirmation; sending someone a login-URL via email actually has less friction because the password-choosing step is removed!
> it's perfectly possible to store passwords in a secure format
But it's very hard to do so. Even if you use scrypt, it is very hard to make sure your whole system is actually secure against password leakage.
The simple truth is that letting your users choose their own passwords is a liability; and I've decided to avoid this liability.
Maybe I'm misrepresenting what the paper states, but my takeaway from it was "don't assume users are dumb when they pick silly passwords. They simply are not willing to use an overcomplex system that for them turns out to be not worth the effort."
I just tried to find this paper online but I can't even remember the title :(
We are told to not re-use passwords. This is not helped by every single shopping web site out there requiring an account (and therefore a password) in order to buy something. Fair enough for big sites like Amazon - I'm actually likely to come back at some time in the future, although I dislike the way it tries to store my card number each time.
On most sites, requiring me to create an account discourages me from shopping there. I'm not likely to come back unless I suddenly have a burning need for another obscure once-in-a-lifetime widget, so why do I need an account? If I do come back, you still only need my card number and a delivery address.
As it stands, the sheer number of accounts that I have means that I invariably set an impossible to remember password and immediately forget it, relying on the password reset mechanism. This is not ideal.
Password reset without the password. If my email account is compromised then everything is screwed, but with password-reset emails that was already true.
Of course, this is potentially vulnerable to abuse... but again, password-reset emails have the same problem.
So HorseBatteryStaple sucks as a password, along with anything else you can easily remember. If you want security, you probably want 2 factor authentication and a different password for every site, probably stored in something like a KeePass DB.
Also, know your target. If your target is to secure your account against a web-based brute force, as depicted in the comic, the attacker is likely to be rate-limited by the server, and a reasonable password is likely to be sufficient. If the attacker gets access to the hashed password database, then that's a different matter, but if you have sufficient entropy in your password it can still be secure.
But my main point is this - why do I need an account and password for uncle bob's glass cutting tool emporium, when I am only likely to make a single order in my lifetime? If I don't have an account, and therefore have no password, then there is nothing to hack.