Likely the most painful password hashing discussion you'll ever read
scott.arciszewski.me
scott.arciszewski.me
In order for a server to use encrypted passwords for authentication, the server must have access to the decryption key -- which means an attacker has access to your decryption key if they can compromise your box. Furthermore, if they don't get your decryption key, but they get your database dump -- they can then just brute your db and decrypt the passwords. They can do this because encryption is, by design, reversible. Encrypting passwords is almost as bad as storing in plain text.
Cryptographically hashing a password is the only way to go. When you cryptographically hash a password, using a known library that has been tested and written by actual cryptographers with passwords/authentication in mind (like bcrypt, etc -- no, sha's and md5's are not safe to use) - you get a one-direction algorithm in that a password goes in, is digested, and a fixed-length hash output comes out. There is no way to reverse this hash back into the original input. Adding salt(s) combats people making rainbow tables against your implementation, as they then would need to know not only which algorithm you are using, how many iterations it does, but also what your salt(s) are (sometimes even varied every so many iterations). This makes it almost impossible for someone to brute your leaked password database.
Nobody has explained this better than Coda Hale:
http://codahale.com/how-to-safely-store-a-password/
(in place of "bcrypt", you can substitute "PBKDF2" [weaker] or "scrypt" [stronger], but all those options are just fine).
SHA256 can be the basis of a scheme to securely store passwords. There are many different names for that:
* key derivation function
* secure password storage scheme
* password scrambler (PHK's term)
* (...)
A secure password storage scheme uses a cryptographic primitive like a cryptographic hash or a cipher to increase the time it takes to decrypt passwords. That's a good thing, because when it takes your user 0.5 seconds to go from "right password -> on disk hash", it'll take a prohibitively long time for an attacker to guess through passwords.
If you mean they call SHA256(SHA256(...48 more calls("password" + salt))?
That's not remotely good enough.
http://hashcat.net/oclhashcat/ -- look at the hardware and see for yourself; those values for SHA256 are million hashes per second.
So 50 rounds of SHA256 can be computed in less than a microsecond.
https://twitter.com/hashcat/status/349192539443699713 -- now, Bcrypt.
I'm not sure what their bcrypt difficulty is, but whatever it is: bcrypt makes it five or six orders of magnitude harder. And remember, bcrypt (like good KDFs) is tunable, so you could make that as hard as you want.
EDIT: Some further consideration: http://arstechnica.com/security/2012/12/25-gpu-cluster-crack...
> As a result, the new cluster, even with its four-fold increase in speed, can make only 71,000 guesses against Bcrypt and 364,000 guesses against SHA512crypt.
https://github.com/freedomofpress/securedrop/issues/180 has some more numbers as well.
And its work factor is adjustable at run-time for the time that hardware speed increases.
Any better? (EDIT: Still in process of reading links.)
Sorry, I'll try to stop incessantly editing ;-)
> Any better?
No.
General purpose cryptographic hash functions are designed to be incredibly fast.
http://www.intel.com/content/dam/www/public/us/en/documents/...
But convince yourself. Do you have Ruby? http://pastebin.com/zjLiYt2j
If I bump up your password scheme by 4 orders of magnitude it's still under <.06 seconds on my machine. And that's with a ridiculously inefficient password generator I banged up in ten seconds.
Use bcrypt. Use scrypt. Use PBKDF2. Don't roll your own password stretching function, it's a recipe for disaster.
OWASP does make a decent recommendation, maybe that'll help give you a leg to stand on: https://www.owasp.org/index.php/Password_Storage_Cheat_Sheet...
PCI is a joke. The self-fill audit: "Are you secure?" -- Uh... sure?
The various scanning companies are of differing quality too, adding to the difficulty in getting any real security improvements from the standard across all ecommerce shops.
> Release v1.0 of super cool app.
> v1.0 gets PCI certified
> Someone discloses a vuln in v1.0
> Release v1.1
> Companies are stuck with v1.0 for at least 90 days until v1.1 can be certified
The problem with md5 and sha and such is that they are fast. bcrypt is useful because it takes a fixed amount of complexity which can be specified by the programmer (and later increased). It also uses salts to thwart rainbow tables. The complexity bit is important, though, because as computers get better, you can make it harder to reverse your passwords.
Also, the biggest reason any of this is a problem is because password reuse is rampant. You are being entrusted with the password to your user's life - their email, their bank account, their facebook account... treat it with respect.
Is there any reason most programmers should know any of this ?
The code should look more like this(and should use a standard library):
import Password_protector
P = Password_protector(date_of_first_use) #library upgrades algorithms every once in a while, only a single algorithm available per date. If there's a need for automatically upgraded hashing library should handle covertly.
protected_password = P.protect_password(password)
if P.compare_password(password, protected_password) == True ...
Or something similar. That at least should cover the basics - safely.
In an ideal world, programmers shouldn't have to worry about this stuff. But this is not an ideal world.
What you need is a password hash, not merely a cryptographic hash.
I'd recommend upgrading to follow best standards. If you use PHP, scrypt isn't that hard to set up.
The big problem you are trying to solve is "our password database just leaked; how fast can attackers figure out the passwords?" Say you can run SHA in 1 microsecond, and 1000 iterations in 1 millisecond. Even though it's 1000x slower, that's probably an unnoticeable difference as far as your performance is concerned. (YMMV)
But, anyone trying to brute-force the passwords out of your database will now have to do 1000x the work.
BTW, none of this will help you much if your users choose passwords like "abc" or "password".
(IIRC scrypt lets you blow up in size as well as CPU, which makes GPUs impractical for attacking.)
The purpose of systems like PBKDF2, bcrypt, scrypt, etc. is to intentionally slow the whole thing down so that it takes on the order of one second to test a candidate. This is a minor burden for normal use, since you don't check passwords very often, but it makes it vastly harder for an attacker to crack a password.
Edit: just to put some numbers on this, it looks like high-end GPUs can brute-force about 2 billion SHA-256 hashes per second. That means that an eight-character alphanumeric password could be recovered from a SHA-256 hash in less than a day, on average. Add 20 non-alphanumeric symbols to your character set and you're still only looking at about six days. Eight alpha+num+symbol characters isn't a terribly great password anymore but it still shouldn't be recoverable if your password database leaks.
Here is some more info: http://forums.udacity.com/questions/6016855/hashing-password...
But are the weaknesses in MD5 relevant to hashing someone's 17 character password?
The problem with MD5 is that it's possible to alter a document such that the forged document still has the same digest. This is a problem for digital signing.
That doesn't seem directly relevant to password hashing. Nobody wants to find some 596 character string which has the same MD5 hash as your 17 character password, even if it does work in its place.
Your password isn't a document that is being digitally signed by its hash for the sake of certifying its authenticity. Nobody is even supposed to store that document!
(Someone tell me where I went wrong, if so.)
The reason why MD5, 3DES, and similar algorithms aren't really recommended in 2014 is that they're very fast/efficient on modern hardware. This is due to utilising GPU cores to do the hashing (which wasn't done back when MD5 was popular). This means the "time to crack" is low.
Here is benchmarks of different hashing algorithms (faster is WORSE): http://thepasswordproject.com/oclhashcat_benchmarking
But the really nice part about things like BCrypt and PBKDF2(Rfc2898) is that they accept "rounds." Which is number of repeated iterations over the password+salt. So every year you can respond to faster hardware by adding more rounds (e.g. 2014, have 10K rounds, in 2020 have 15K rounds, etc). If MD5 had supported rounds, it might still be safe to use today with enough iterations.
Even things like BCrypt and PBKDF2 will eventually fall by the wayside (MD5 used to be a more secure alternative to the much faster DES algorithm). But at least rounds (additional iterations to respond to changes in hardware) will keep them relevant for longer.
You use a salt to mitigate this but even if you use a salt, it _has_ to be stored somewhere. Once an attacker has it, they'll be able to create rainbow tables with this in mind.
You then move to using a different salt for every password so any rainbow table only works for one result but if you use a fast hashing algorithm, attackers will still be-able generate rainbow tables based on those salts.
So you pick a slower hashing method but in the future, as computer hardware advances, you'll be in the same situation where it's trivial to brute force some rainbow tables.
So you need to keep on top of it and keep updating the stored hashes with something that is non-trivial to brute force, in PHP this gives you these functions: http://php.net/manual/en/ref.password.php.
In reality though, in the event of a breach, all you're doing is buying time for your users to change their passwords before the relevant rainbow tables are generated, so the next important step is to let your users know their hashed passwords were leaked, you followed best practices, but they should update the passwords wherever they used it.
A SQL injection vulnerability lets you get the MD5 hashed passwords out of a database. As most people don't use 17 character passwords, you can check the MD5 hashes against a rainbow table (you can even do it online) and probably find a privileged user account pretty quick.
I actually broke into a legacy system my predecessor programmed in this manner when I couldn't find documented admin credentials.
MD5 is fast - and while computational speed is normally your friend, with cryptography it's your worst enemy. Simply put, the faster the cipher, the more break in attempts a cracker can have per second.
This is why password strengths are often represented in duration (ie how many years would it take to crack). Though those figures are hugely estimated - to the point of being practically worthless.
An interesting feature of some dedicated password hashes is that the salt input can contain encoded parameters, such as the number of rounds or maybe even choice of algorithm. This means that we can "future proof" a hash to some extent, while allowing it to support existing old passwords, while increasing the number of rounds for new passwords. A plain message digest function doesn't have these features, needless to say.
Collisions are the most practical method of attack on password hashes. If you develop a way to find collisions faster than an exhaustive search of the key space, you shorten the time it takes to find a password that matches the hash. Collisions have now been found on MD5 very quickly (on the order of hours).
Preimage attacks are not all that useful when password cracking because you don't need to generate a new message that hashes to the same hash you have now, you're looking for any message that hashes to the hash you have now.
Second preimage attacks aren't relevant because they assume you know the password already and are looking for a new message/hash that match.
KDFs are meant to require a great deal more computation (e.g. an example of PBKDF2 might iterate HMAC-SHA256 8,000 times).
It should be: "do not use any un-iterated message digest function for passwords: password hashing functions iterate a basic hash through numerous rounds, perhaps thousands; and some even encode the number of rounds as part of the salt, so the rounds can be increased in the future without changing the function, so it still recognizes old passwords."
If your 17 characters are 3 lowercase non-archaic English words, maybe with two digits at the end, then a codebook for them already exists in MD5.
In practice, we don't know how crypto analisys will evolve, and non-broken algorithms are free for use, and just as easy as MD5. So, why do you insist on using a broken one?
I'd say this has to be a joke, but Poe's Law is a tricky thing.
Wow. So MD5 is fine as long as it isn't needed to protect the data. Looks like a typical PHP discussion thread: plenty of misconceptions, a mix of of bad and plain stupid advice with a pinch of SQL injectable code. Silly PHP developers.
- Bad advice is posted everywhere.
- Good advice is disregarded because nobody reads the preceding posts in the discussion.
The question was "I want to store password[s] in encrypted form." The answer should have been immediately -- "you don't encrypt passwords, you hash passwords using a well known, established, and tested cryptographic hashing algorithm + a salt, damnit!"
Your answer is as wrong as anything posted on that discussion.
The correct answer is that you don't encrypt passwords. You also don't hash passwords with a salt.
Instead, you use a key derivation function such as PBKDF2, bcrypt, or scrypt.
If you are not using a salt, then you are vulnerable to rainbow tables which contain a few hundred thousand of the most common passwords. Even if your hashing algorithm is slow and complex, there are still rudimentary rainbow tables for them. If you haven't used a salt then with one quick scan at least of few of your passwords will be revealed.
The effect of a long random salt is to make every password appear to longer to the hashing algorithm. What's being hashed is the users password plus 20 random characters, and nobody has a rainbow table that contain those 20 random characters. If your hashing algorithm, like bcrypt, accepts the salt as an argument, then that makes things all the easier.
You should not hash passwords. You should never hash a password.
You should use an appropriate key derivation function (bcrypt, scrypt, PBKDF2). Key derivation functions are designed to address the issues you note and many more.
Now, let me give a few choice tptacek quotes on this:
>The socialbookmarkosphere is abuzz with talk of “rainbow tables”(...) This really freaks me out. If the “advanced” pole of your threat model is “rainbow tables”, stop working on your social shopping cart calendar application right now: I can’t trust you with my Reddit karma score, let alone my credit card number.
> Rainbow tables are easy to beat. For each password, generate a random number [... and do what you said to do ...] Cool, huh? Yeah, and Unix crypt —- almost the lowest common denominator in security systems —- has had this feature since 1976. If this is news to you, you shouldn’t be designing password systems. Use someone else’s good one.
---
I'm banging on this drum for a very simple reason: even if you are smart enough to know that salt+general purpose cryptographic hash function isn't good enough, when you say "hash" programmers that aren't smart enough are getting the wrong message.
And if you do believe a "hashing algorithm with a long random salt" (which I translate to "general-purpose cryptographic hash function, such as SHA/RIPEMD + a nonce") is sufficient to store a password... well, let's hope you don't.
"Therefore all KDF functions are cryptographic hash functions. All hash functions are not KDFs, due to speed."
Using that term to refer to a KDF is exceptionally misleading and foolish.
If you gave that answer to those PHP developers, they'd come out thinking MD5 and a salt would Work Just Fine.
The important property it has that other non-cryptographic hashes do not have is speed (actually, a lack thereof) and (typically) the ability to tune that speed.
If we want "technically correct" points I won't grant them (;-)) because KDFs have a property (slowness) cryptographic hashes do not. It would be technically correct to state that KDFs are strict supersets of cryptographic hashes.
Practically speaking, calling a KDF a cryptographic hash is very harmful because the average programmer doesn't have the understanding to make the distinction. It's important to give them two very clearly separated categories - cryptographic hashes are {SHA1, RIPEMD160, ...} and are FAST, KDFs are {scrypt, bcrypt, ...} and are SLOW - and a clear rule that "you MUST always use a KDF to store passwords".
If you blur the lines between the two, the average programmer will misunderstand and use the wrong tool for the job.
Why would the speed of the hash function make any difference, assuming it's a cryptographic hash? If it's faster, run it for more iterations - if it's slower, run it for less. You pick a maximum time, figure out how many iterations can be run in that time, and go from there. I would have thought that a fast hash function would be better, if anything - as it's potentially already more optimized for current hardware, which means that it's less likely that it will get faster on better hardware.
I always thought it was the memory complexity of the hash function that was the difference, not speed. Makes it harder to run on GPUs / ASICs.
I'm gently curious if black hats will now target some of those companies?
Or are there enough low-hanging fruit available from www searches for them to not worry?
Indeed too many people there failed.
Then I noticed "PHP" and then I was enlightened.
/s
Then later credit you for your advice.
I was on your side for awhile, advocating that we just teach people about what a KDF is, but my side lost.
Do you have any specific examples of bad security advice in the documentation? I can try to raise hell with the doc maintainers and maybe even suggest an amendment.
> Exploits for the MD5 and SHA-1 algorithms have become known. You may wish to consider using one of the other encryption functions described in this section instead, such as SHA2().
Further down:
> The PASSWORD() function is used by the authentication system in MySQL Server; you should not use it in your own applications. For that purpose, consider MD5() or SHA2() instead. Also see RFC 2195, section 2 (Challenge-Response Authentication Mechanism (CRAM)), for more information about handling passwords and authentication securely in your applications.
1: http://dev.mysql.com/doc/refman/5.5/en/encryption-functions....
Related, the only hashing algorithm supported by postgresql for authenticating to the pgsql server is md5.
> If you are at all concerned about password "sniffing" attacks then md5 is preferred. Plain password should always be avoided if possible. However, md5 cannot be used with the db_user_namespace feature. If the connection is protected by SSL encryption then password can be used safely (though SSL certificate authentication might be a better choice if one is depending on using SSL).
Strongly implies that md5sum offers protection from an attacker sophisticated enough to sniff passwords of the network. Snif.
Anyway, the password will be stored in plain-text just a few servers away. It has to be. Whatever hash they use, hashing it will be as useless.