Password leaks bigger than first thought | Hacker News Reader

Password leaks bigger than first thought

h-online.com

33 pointsbond52 comments

"And one amusing detail – although eHarmony implores its users to use strong passwords including both upper and lower case letters, it saves the passwords in all upper case"

This is truly beautiful - made my day :)

Unsurprising that >95% of the password hashes have been broken. I remember being annoyed when I signed up for LinkedIn (just checked my tweet history - it was 2010/06/10) because they were only allowing 16 characters in the password field.

EDIT: Whoops, guess I should have done some napkin math before claiming that there are rainbow tables that cover that area. /me slaps wrist

26^16 ~ 4e22, so no such rainbow tables exist.

I mean... there are surely not rainbow tables that span all 62^16 16-character passwords consisting of alphanumerics only: that is 646,081,519 zettabytes of hashes (62^16 * 128 bits).

There are clearly rainbow tables which span a more useful dictionary of passwords, though--and certainly all the most common passwords.

Even if we conservatively assume that you only need lowercase characters, and that you need one bit per password, that's still 26^16 bits = 4.6 ZB, about fifteen times the total capacity of every hard disk sold by Seagate in 2011 (330 EB).

They usually use less than one bit per password.

People here don't know what rainbow tables are. If you asked them to build a rainbow table, you'd get... a table. (You're right.)

Thank you for the contentless comment. Instead, you might consider offering a more interesting algorithm, its tradeoffs, and why it is beneficial to take more computation time?

A rainbow table doesn't store every hash for the password space it's built for. I'm sure you already knew that, but decided to pretend otherwise when calculating the size of one.

You are absolutely correct; I should have actually done the 5-second calculation before writing that instead of relying on my flawed memory.

Looks like the publicly available rainbow tables go out to 7 characters with special chars included, which comes out to a much more reasonable 3.2 TB (using 62 chars).

But still a good point: Why limit your password length? My online banking is the worst offender that I know. Not only do they insist that the password is no longer than 8 characters, it HAS to be 8 characters. Madness!

I agree, banks having this flawed security is quite frightening!

My bank used to have an 8 character password limit policy, but recently changed it (without announcing it), and I was able to use a 20+ character password. It's worth re-trying once in a while just to be sure.

Good idea, will try it out next time I'm in it.

Last month Last.fm admitted to having received several reports of spamming involving user data.

Over the last 6--9 months I have definitely noticed an uptick in the random "connection" requests I get on LinkedIn. I don't know if this is because their userbase has grown and more people are just shotgunning connection requests, or if these represent first steps at an attempted social engineering attack via hacked accounts on which I appear in the "people you might know."

So far none of these have been from people I actually know even remotely, so I'm guessing it's just simple spam (and I report it as such).

I find it really incredible that this companies were so careless. Really. I know that security practices are rare to come by, but come on! LinkedIn, eHarmony and last.fm! These are some of the biggest websites.

I would be very surprised if this sort of problem is not quite widespread among companies with giant userbases.

We'll see more of these, and bigger ones.

Microsoft Active Directory servers store passwords as plain MD4 hashes. http://16s.us/ms_ad_hashes/

These servers are used by governments and large organizations all over the world.

I would like to assume that the REALLY big companies like Google, Microsoft, Apple, Amazon, etc. can figure out how to effectively store passwords. While I acknowlege that Active directory is a bit different from a web service, if this can't be realized, god knows what's happening with the other 500 sites we all have accounts on (that invariably use shared passwords: http://xkcd.com/792/ ). I really love it when I get password reset emails with my original password in plain text.

Exchange email systems have web interfaces. They typically authenticate with AD.

Discussing hashes internally with Commander Adams, among the reasons we use unsalted hashes for internal passwords was that Google Mail for Domains (with which we use an SSO tool) doesn't work with salted hashes.

Or should I be siccing Krell monsters on the good Commander again?

Yea it dates back to the beginning of NT (which is why it is MD4) and by now this hash is used in so many Windows protocols it is hard to change.

I'd argue that security practices are not that hard to come by:

https://www.owasp.org/index.php/Password_Storage_Cheat_Sheet

I meant that are hard to see implemented on websites, not hard to find. Sorry

This page, when I read it a couple days ago, was pretty bad.

What reference would you recommend instead?

Coda Hale's page.

http://codahale.com/how-to-safely-store-a-password/

Let's be clear-eyed about this: we're talking about an OWASP page that says the "1-2-3" rules to do passwords well are "use SHA256", "pick a cryptographically random salt", and "iterate the hash".

Well, #1 is a suggestion that makes virtually no difference; your outcome won't be meaningfully worse if you use the (broken) MD5 algorithm than it will be if you use SHA-2.

And #2 is also a suggestion that makes virtually no difference; the (relatively unimportant) attack that "salts" blunt does not depend in any practical way on predicting salt values. Strong salt, weak salt, same deal: rainbow tables stop working, everything else still does.

Rule #3 is the only thing that matters on this page, but, of course, simply iterating your hash 64,000 times is an inferior solution to PBKDF2, bcrypt, and scrypt.

This page is a wiki, and I called it out on Twitter a day ago so the odds are some of this stuff has been "addressed"; but, I thought about taking a stab at fixing up the page and realized I'd be trying to incrementally improve that 1-2-3 advice. And, to the specific point of this thread: whatever better advice is there today, it wasn't there last week for LinkedIn or eHarmony to take advantage of.

Unsalted hashes. Wow. What a bunch of amateurs.

Also, 10 internet points says at least one other major website will fall too within 2 weeks. Someone has found a new exploit & is trying various sites & collecting hashes.

It doesn't really matter if you're salting or not when you use really fast general purpose hashing functions. Consumer video cards can calculate half a billion SHA/MD hashes per second nowadays. You're talking minutes vs seconds. Unacceptable.

"The API was developed 9 years ago, and appears not to have been updated since."

Last.fm could have updated this, except it would have meant making all their users do something.

Why not use lazy rehashing? On login, the password is availible and so can be rehashed properly.

Mainly because many Last.fm scrobbling clients relied on the legacy API that uses a person's password as a private key to authorize requests.

They send a token hashed with the password and they have to keep the original md5'd password on file in order to allow these clients to work.

You can do better - if they're storing MD5s of the passwords, all they need to do is hash those again with another salt:

    BCRYPT(MD5(Password))

Running BCrypt or SCrypt over the current MD5 hashes is easy, and they can do it right now for every password. If someone (else) grabs the database in ten days time they get no MD5 hashes of passwords instead of half of the userbase.

I believe the legacy api required md5(md5(password) + time) or something like that. Which means they needed to store the md5 of the passwords or modify all third party clients that used this method.

What do people think about outsourcing your authentication to someone else?

Full Disclosure: I'm currently working on a brandable authentication host (http://www.authic.com) that will outsource the pain of storing your password hashes securly and provide your web app with slick a user account UX.

I'm offering my users Browserid and/or Facebook in order to login. Works great and passwords are definitely something I'd rather not deal with. Also saves me from having to implement all sorts of stuff like forgot password, forgot username, etc.

Do you also offer your own login forms?

Nope. Feel free to play around and try it yourself. I think the experience is not 100% perfect but is pretty darn good. That said, I'm always open to constructive feedback. www.voo.st

Looks nice. My biggest feedback is that there is no "Join now" link/button, just a login. I presume that if you try and login without an account it will go ahead and sign you up, but it is not obvious.

Also the line "All events on Voost are managed by an organization" - I'm not sure what that means, what is an organisation? Is that me?

Great feedback, thanks. The button is 'sign in' since we don't really need the concept of login. Adding another button seems more confusing, what would it do?

As for the text, sure, we can work on clarifying that more.

Having two buttons doing the same thing wouldn't necessaryily be confusing. People would probably not notice.

Or how about change the wording to "sign in/up".

We all do it daily with Facebook et al delegated authentications.

My own personal attitude is that I'd prefer having independent, unlinked accounts among various service providers, for which I use long, randomly generated passwords.

My risk from any given service is relatively low. My passwords are strong enough (trillions to quintillions of years brute-forcing time per the online calculator's I've checked -- with similarly constructed passwords, not my actual ones, natch) that risk of bruting a hashed key is low, and I don't re-use passwords. For services that store passwords in cleartext (still fairly common practice on mailing lists), no big loss either.

Does the number of passwords in the hashed list matter in terms of how easy or hard they will be to crack? Does this have implications for a rainbow table-type attack?

I guess the more you have, the more information you have about the salt used. I'm not sure it'd be useful though.

And many of these hashes aren't salted anyway.

The real question is, if a site used a salt, was it a per-user salt or a global salt?

Often people will store the password salt alongside their password hashes*, so if someone has the user table they would have the salts.

---

At Authic.com we are investigating the practicalities of storing salts in a separate database from the password hashes.

Doesn't that still leave the door open to database compromise?

What are the practicalities of distributing the system among various technology stacks, minimizing the damage caused by a 0day or single engineered attack vector? Really thats just security by obscurity and the solution should be a robust system to begin with but I'm wondering about anticipating the human mistakes that render an otherwise secure system into a series of news articles like we're seeing.

Doesn't that still leave the door open to database compromise?

Well, if you have it in the same DB table you are still at the mercy of database compromise. Better would be to store the salts in a separate table from your password hashes. Better yet, store it in a different database. And even better, store it on a different server.

What are the practicalities of distributing the system among various technology stacks, minimizing the damage caused by a 0day or single engineered attack vector?

Yes, having the salt in a different type of db from the hashes adds protection from zero day exploits exposing one brand of db. A combo of NoSQL and SQL seems logical.

Really thats just security by obscurity

Security through obscurity is not a bad thing, as long as it is not the only measure you are taking. But anyway, you need to have security through obscurity - or are you suggesting that making your salts and hashes publicly accessible would be ok? Or is it a good idea to obscure it?

and the solution should be a robust system to begin with

Of course, you should do everything possible to make system secure. A big part of this is to try to minimize the number of single point of failure attack vectors.

but I'm wondering about anticipating the human mistakes that render an otherwise secure system into a series of news articles like we're seeing.

Well, a lot of these news articles could be lot less worrying if people were salting their password hashes properly.

Security by obscurity is the only kind that exists.

Just use a hardcoded application salt (significantly large, >128 bytes is more than enough) in addition to your per-user salt (stored in the DB, or derived from a DB field). That way, an attacker would need both code access and DB access to crack the hashes, and if that happens you're basically screwed beyond redemption anyway.

We really want to go the extra mile with Authic (it's going to be main selling point of outsourcing your webapps authentication with us), so we'll consider adding as many different aspects as possible.

And yes, you do have big problems if someone has access to your db AND code, but if you have done your job properly at least it will be very difficult/expensive/time consuming for the attackers to crack your hashes.

In the worst case scenario of someone dumping your entire database, you want there to be as much time as possible to contact your user base to let them know of the breach so that they can update their passwords before the crackers have finished the job.

It would increase the likelihood of more common passwords being present in the list giving the attackers insight into what hashing technique was being used to store the passwords.

But the difference between 10,000 hashes and 10,000,000 would not make it easier to crack.

Reply on news.ycombinator.com