2,800 New Data Breaches with 80M Records Added | Hacker News Reader

2,800 New Data Breaches with 80M Records Added

troyhunt.com

201 pointsweinzierl96 comments

Just let that number sink in a bit. 80M/2844 = ~30K accounts / breach.

And still, just about every service that you have to make use of (your HOA, your insurance company, your government and so on) wants you to trust them with your data secured by a userid and a password that they will administer on infrastructure that they are most likely not competent to secure.

One of the best things to come out of the GDPR is the reporting requirements for breaches. You can expect a lot more of these updates in the future.

Edit: interesting, the link now goes to 'https://ghost.org/fail/', a 404, so go to https://www.troyhunt.com/

It's amazing how little the breaches matter, too. We like to gnash our teeth whenever a breach happens, yet somehow the breaches matter less than, say, getting into a fender bender.

To a point. Imagine if some crafty hacker leaks Slack's entire message history across all companies. Think of the chaos.

Kind of weird to consider that maybe we haven't seen a catastrophic breach yet.

Ashley-Madison was a pretty serious breach in that respect.

People died because of that one:

https://en.wikipedia.org/wiki/Ashley_Madison_data_breach#Imp...

Just to add on top of that the OPM breach was and still is pretty serious. A lot of personal information for a lot of people was leaked and is easily in the hands of malicious state and individual actors. The sheer scale of the breach and the amount of information out there can be leveraged by a lot of people.

[0] https://en.wikipedia.org/wiki/Office_of_Personnel_Management...

[1] https://www.wired.com/2016/10/inside-cyberattack-shocked-us-...

[2] http://www.slate.com/articles/technology/future_tense/2015/0...

The impact is often not seen immediately as databases of leaked accounts can be used for further hacks (impersonation, social engineering, blackmailing etc.) I'd attribute that to the lack of studies and difficulty to trace hacks back to leaks.

The link on Troy‘s Homepage gives me the fail page also. Seems to be a problem on the Blog page and not on HN. At least I tried the link when I posted to HN, and it worked then.

I wrote Troy a message, but I think it’s around midnight in Queensland and he might be offline.

You're right, now they all fail. I have this ugly Google cache link for you:

https://webcache.googleusercontent.com/search?q=cache:jLWeTh...

If you look at the list of the 3000 breaches in the blog post, these are mostly small websites. I am sure the majority of those are still unaware they are breached.

I remember a few years ago creating an account on a small online clothes shop with a unique randomly generated email. I almost immediately started receiving unrelated spam on that email. Clearly their database had been breached. I still haven't received any notification on haveibeenpwned for that email. I am sure there is an enormous mass of breaches of small mom and pops sites that are out there, unknown to both the users and the owner of the sites.

Yes, I would not be surprised at all if the majority of breaches went undetected, which means the number of them that are unreported will be very large as well.

I really do not like making accounts, especially where they are not required, such as when ordering something from some shop that I only intend to use once, or for my mobile phone company, who in their infinite wisdom have decided that having my email address is not enough to send me invoices, they need me to receive an alert that they have posted an invoice in their internal system for which they helpfully created an account that they mailed me the password for.

It really makes me wonder who comes up with these ideas.

John here from ghost.org/fail - not sure what happened there! Can't reproduce, but sounds like it was a temporary DNS resolution glitch. Will keep an eye on it

In case this helps to debug the problem: there was a Cloudflare interstitial page for a bit before it started to redirect to the 404 page.

Also, I just re-tested and it is still not working.

Yeah, sounds a lot like a CF DNS resolution problem at one particular edge node. Are you able to stop the browser on the CF error page and copy/paste the RayID for me? Would be helpful for us to report upstream and ask CF to flush their DNS cache at that location :)

Ray ID: 3f3437dbbdab79ff

Your IP address: 1.2.3.4

Error reference number: 524

Cloudflare Location: Seattle

Ray ID: 3f343e54e87a9cbf

Another possible hint: opening the link in a FF private window works.

I worry we're approaching a situation where everyone just assumes their data is out there, lethargy / ignorance of the users sets in stone, and as on the whole we don't make any progress security wise.

It will change when those people suffer very real consequences of the endless attempts of data mining -- such as being denied health insurance for something you posted on facebook.

I don’t think we’re in that situation yet. I think the average person is quite surprised what can be found about them by Googling their own name (or user handle) and looking past the first pages of Google results. How many people assume that their familial relationships and all their past addresses is easily findable using one of the free people-search services?

We're not approaching it. We're past it. Everyone's data being out there, that is.

So my most-breached e-mail address (a hotmail address from 17 years ago that I basically use for stuff I know will send me spam) has 3 or 4 newly-added breaches.

  2,844 Separate Data Breaches (unverified): In February 
  2018, a massive collection of almost 3,000 alleged data 
  breaches was found online. Whilst some of the data had 
  previously been seen in Have I Been Pwned, 2,844 of the 
  files consisting of more than 80 million unique email 
  addresses had not previously been seen. Each file 
  contained both an email address and plain text password 
  and were consequently loaded as a single "unverified" 
  data breach.

  Compromised data: Email addresses, Passwords

The problem is I have no idea what this breach is sourced from, so I don't know what account in particular was affected. Not Troy's fault, obviously, but it's basically just telling me, "Your e-mail address and a password to somewhere was found online somewhere." For all I know it's recycled data from a breach I already know about.

At this point it's not really a big deal. I use a password manager and they're all random at this point, so I just need to figure out which one to rotate.

Same. I got the email this morning and looked to see what I need to do. Turns out, what I need to do is just, like, be wary? No way to know what accounts I ought to go change passwords for. Seems unreasonable to go change literally all of them over this, that would take forever. This news makes me panicky but gives me no reasonable course of action.

I wrote a little python script to match the websites with the Alexa Top 1 Million and sort by rank. I found 268 top 1 million sites in the pwned list. You can check out the ranking here if you want https://github.com/tail-recursion/hibp-rankings/. Notable websites include; lotro.com, mtgox.com, malwarebytes.org, daemon-tools.cc, autohotkey.com, vbulletin.org

So, recycled old dumps all of them? Every single big site I can see here has coincidentally already had their data publicly dumped.

It's getting to the point where it feels like the site could just be replaced by one of those joke sites where the answer is just a static string: Yes.

Hopefully at least it will push more people towards password managers and avoiding password reuse.

This information isn't valuable in any way to affected users without an easy way to verify. Obscuring that information doesn't protect the users, it only possibly sends them to places on the web where they are at a higher risk for some other security breach. This is just click bait.

I kind of agree. This Troy Hunt guy is doing a fantastic job with HIBP but this particular release is just so nebulous it's a real time waster.

What information are you referring to that is he obscuring?

For example, if the names of the files included in the zip file may have been a domain name. From seeing the contents of these dumps before, files are often named in that manner. If they are, then I as the user can then determine where my account was breached.

This of course is just one example, and not guaranteed to be an accurate one in this case.

Something more than the list at the bottom of the post?

Adblock caught that - apologies.

When an email comes up as compromised in this dump, it would be great if that notification came with the filename(s) in that list where the email was found.

Especially in this case I think including the filename would have been nice; manually going through list of 2.8k domains and trying to guess if you ever have happened to have an account there is no fun. Having the filename might also help verify the veracity of the information.

I suppose the reason for omitting the filename is that Troy does not want to cast any unwarranted trust in the filenames, which is fair.

My first guess was that by publishing the file-name, it is easier to link a compromised email to a password. I'd guess that, if you had access to the dumps, this functionality would be about a 10-20x speedup to lookups of the password belonging to an email.

However, I think the real reason is privacy. If I recall correctly, there are certain breaches where HIBP redacts the website to protect privacy. A great example of this was ashleymadison. It kinda sucks when anyone that knows your email can learn you were on that site. It seems excessively hard to screen the entire list of 2800 items for such privacy concerns.

he publishes the list of domains in this article. I've already recognized one i had an account on, and was able to notify the owner - it looks like it might have been from a known 2016 hack

not sure why you say that. If you subscribe to HIBP, and your address turns up in one of these breaches, you'll get a notification.

you can also search now for your addresses and see if you were affected.

Problem is, the pastebin has been removed, so I don't know what account associated with my email address has been compromised.

In this case though, HIBP will only say that your email address appeared in any of the files - it won't tell you which one.

Simply knowing is usually good enough for me - nearly anytime my email is identified in a potential breach, I assume it is time to check my accounts and roll my passwords.

And just because XYZ site was previously listed as a breach, doesn't mean it wasn't breached again with your new password

shouldn't you just periodically check your accounts and rotate passwords? This is obviously not a comprehensive database of every breach.

Yes, but you gotta find a balance - you'll go crazy if you rotate passwords everyday! Seeing your email in a new breach is certainly a good time to do so.

Having your address in the list + having your password in https://haveibeenpwned.com/Passwords is a safe bet.

I'm a little surprised he doesn't share actual password lists with at least some other security researchers. I get there are legal issues involved but I thought researchers had legal protections?

"Dark web" (lol) hackers are already going to be circulating the sources of this list and in any case could probably crack sha-1 reasonably quickly. So if you do a minimal amount of verifying the person who asks (e.g. ring their university) you can surely help the good guys with little risk of misuse?

https://haveibeenpwned.com/Passwords

They're SHA-1 hashes.

Two comments ago, you:

> in any case could probably crack sha-1 reasonably quickly.

So that shouldn't bother you then.

As I said, that shouldn't bother crackers. It does bother security researchers who have much less in the way of resources.

There's half a billion hashes. I don't see how anybody could crack them "reasonably quickly".

Most of them will be known already. It's mostly just a matter of comparing hashes or at worst converting plaintext SHA-1 (which is designed to be very fast).

And crackers will share unknown hashes to be cracked. So they can crowd source cracking far more effectively than the security community can.

That suggests the security community is less organized and has fewer resources than the crackers.

> That suggests the security community is less organized and has fewer resources than the crackers.

That suggestion likely reflects reality.

In addition, the list isn't connected with any specific accounts* - even if you broke the whole list (which it essentially is already, anyway) someone would have to match each one to some rando account. Not likely.

*unless you have the original breach, then we are back at zero: its already compromised.

Well exactly. It's of minimal (if any) use to crackers. But analysing cracked passwords could be of use for research.

500m salt-free hashes just mean your odds of getting a hit per hash go way, way, way up. Around one in 2.9e39 instead of 1.5e48.

He recently shared Pwned Passwords which lets you check if your password is in the breached data.

1Password then did a POC integration into their service so you can test if your password shows up in HIBP data.

https://blog.agilebits.com/2018/02/22/finding-pwned-password...

Right but that's not so useful for security researches. Analysis of the actual passwords would be more useful.

His reasons:

https://www.troyhunt.com/no-i-cannot-share-data-breaches-wit...

https://www.troyhunt.com/here-are-all-the-reasons-i-dont-mak...

Indeed. I guess the best practical argument is sharing them securely is hard. Which is very true. However if it's not possible for two security researchers to exchange data then what hope is there for the rest of us?

I get he doesn't want to do that and that is his prerogative. However, it does feel like we're so scared of things falling into the wrong hands that we hobble our ability to defend against hacks in the first place.

I would imagine if there are any security researchers he trusts sharing the data with, they wouldn't need to email him out of the blue for it, nor need to discuss it with the rest of us.

But no, there is likely no hope, lol

Like most of you (probably) I have more than one email address.

My host 'hacked' email address has been involved in 6 breaches.

Anyone beat that?

A very old email I've used a long time ago has been involved in 7 breaches and 1 paste. None of these passwords have been valid for years though.

No way for me to verify it because I use different username on different sites, and I have a self-hosted catch-all email server, I would have to extract all my usernames or emails from KeePass and check each of them separately... I'm too safe to even know I was hacked.

That might be even more interesting actually. Seeing as you could identify the exact site that was breached. That sort of data would actually be quite valuable to this project I would surmise.

I also have a similar system now as you do, but really only for higher value accounts, not the niff naff forums stuff. And like you it's all randomly generated unique passwords so pretty safe.

None of my site specific emails have been breached yet, which is good as that would mean a major global provider had been breached.

> That might be even more interesting actually. Seeing as you could identify the exact site that was breached. That sort of data would actually be quite valuable to this project I would surmise.

That's why I do it.

The project seems to know where the data comes from, at least in my case it identified each:

    Email 	 	Pwned sites
    dropbox@... 	Dropbox
    ffshrine@... 	Final Fantasy Shrine
    github@... 	 	GeekedIn
    linkedin@... 	LinkedIn
    patreon@... 	Patreon
    tumblr@... 	 	tumblr

(The Github one was using their OAuth - which kinda screws with the traceability)

Unless you're using unique domains, you could use the HIBP domain-wide checker at https://haveibeenpwned.com/DomainSearch.

I have a setup where I have unique emails for each site I register with, but they are all under a specific domain. I was wondering how I could make use of HIBP and this helps immensely – thank you for sharing!

I think that's what you want:

https://haveibeenpwned.com/DomainSearch

I just checked my own domain and there are three addresses pwned that I don't use and never created. I've had the domain for years too.

Probably someone just making up an address randomly but three times is a bit odd.

7 spanning 6 years. What's interesting, is that is the account I use for the more mainstream sites. The accounts I use for sites I don't trust, all has 0.

Probably just because the more mainstream sites are more 'valuable' targets for breaches.

This goes not just for hackers, but also for Troy.

If some small site gets hacked there is a much smaller chance that it finds its way to Troy.

17 and 2

16 sites (+2 pastes) with an old email address.

Including this breach, I'm up to 13 breaches and 1 paste on my personal email address.

I hadn't checked in a long time, I have 13 now. Wow.

I've got 15 here, but I've used this address for a long time. I've been using unique passwords for a long time too, though.

> Anyone beat that?

Sure: Zero ;)

But that's no real surprise, I try to keep the number of accounts I have to an absolute minimum which is one of the easier ways to avoid this fate.

Hey PuffinBlue, my email is in 14 breaches :D so yeah, I beat that. (my email is in my profile, you can check)

2 years ago I started using 1password and "user+sitename@domain.com" so I'm no longer worried. I assume all accounts that I haven't migrated are compromised.

17 breaches here :)

I ran the emails of a few of my family through the pwned API so that I could tell them to change their passwords. I was surprised that almost everyone I reached out to was surprised by their information being out there. My interpretation was the general public is almost entirely unaware of the scope of these breaches and what they need to do as a result.

It'd be cool if there was some way of gently sharing with your friends and family whether or not they've been pwned -- maybe some kind of social network plugin or web app that generates emails for you to send to them. A key challenge would be being sensitive to their privacy, i.e., not coming off as creepy.

Is there anything like this but for credit cards rather than accounts/passwords?

JFYI: I've seen plex.tv.txt in the list.

line 713

Most likely just the old dump repackaged.

Yep. But I thought it better to say it.

Thanks, I couldn't be bothered to go down the list but was wondering where I was breached. This is probably it.

I received a notification earlier, though only regarding a dead email address that I forward, it's probably some forum I signed up to ages ago and forgot about.

I guess this is also a good time to remind the general population that password managers are a thing and prevent such breaches from turning a "oh I don't visit that website anymore" into "oh sh*t my bank account just went to 0"

I applaud the dedication and effort that the author puts into HIBP.

One small suggestion, though: I wouldn't harm to ask somebody to proofread public statements like this, especially since this is a highly trust-related topic. The density of typos is annoying to me even though I'm not a native speaker.

One of the nice things about changing email addresses... every single notification I've ever gotten from this site applies to my old addresses. A couple years after switching, no breaches of my current credentials have appeared yet.

Link currently 404s unfortunately. Navigating directly to troyhunt.com works fine.

> I think that would be a much more interesting and useful way to spend two hours, rather than implementing cute little algorithms inside an isolated environment like HackerRank.

Hiring manager here, one who doesn't use HackerRank either. I'm in agreement with the premise, but I would likely decline an applicant unironically employing the tone of writing emphasized above given the implications against cooperation and teamwork.

"cute little" can be replaced with "case-specific" or "niche," maintaining the same general meaning while divorcing it from adversarial connotations.

wrong post :')

I'm sure most people here are already familiar with how to make unique and strong passwords but for those who aren't:

https://inspiredelearning.com/resource/create-strong-passwor...

I don't recognise any of the hosts in that list. It's going to keep me awake now wondering which one of them I had an account with that was leaked. Might go find a torrent and do some spelunking ...

Plex.tv.txt? (line 713)

There should be a way to check entire domains, outside of known public email providers. I use a different email address for every website I use and a catch-all rule on my domain to redirect them all to a single email account. I won't check them all simply because I don't even remember them all. I know this could have the downside of being useful in email address enumeration though.

There actually is - use the Domain Search (https://haveibeenpwned.com/DomainSearch)

Thanks for this link, looks like all the emails in my domain are safe.

Reply on news.ycombinator.com