Сhecking for leaked data without storing any | Hacker News Reader

Сhecking for leaked data without storing any

medispank.com

37 pointsgorokan22 comments

Reversing the phone number hashes is trivial - there are maybe 8 to 10 digits in a phone number and something like hashcat will probably try them all in less than a second on a single current GPU. And it is probably not that much harder to reverse most of the email addresses - a suitable dictionary paired with some rules how people choose their email addresses and you are good to go.

Just checked: hashing all possible 100,000,000 mobile numbers on a single CPU takes 14 seconds. I am sure a rainbow table with such data already exists.

While that's true, the phone numbers without the name and other information cannot be weaponized.

The phone numbers can be generated - there are only 8 changing digits in Australian mobile phone numbers (the first 2 digits are always 04).

> the phone numbers without the name and other information cannot be weaponized.

On the contrary - they can and routinely are.

Speculative case:

The space of valid phone numbers is small, the total number of subscribers is peanuts for automation, and bulk calling is cheap. As a result, it's affordable in some cases to just literally call every single number (prefiltered through one of the marketer databases of numbers belonging to real people) and try the scam - people who aren't in your target group will only get confused for a second, and write it off as usual robocall, while those in your target group will either fall for it, or otherwise identify themselves by recognizing the context.

Or at least that's what I believe is happening, based on calls I occasionally get, trying to "give" or sell me some crap from a product/service category I never cared about, available in a city on the other side of the country.

BTW. This is what's constantly happening in the e-mail space. I get tons of messages that attempt to phish people by hoping a percentage of recipients will happen to be customers of a specific company, or awaiting a package from a specific delivery company on a specific day. I personally had a close call with a phish about OneDrive data deletion that hit my company inbox on the very next day after the company migrated my data across corporate systems - by pure chance, the message looked almost perfectly legitimate, for me, on that day.

Common case:

The usual plausible deniability defense of legal scammers (telemarketers) is, "this number was randomly generated by our system", which is both technically plausible and obviously bullshit. However, being on a receiving end, I have neither time nor means to prove they got it from a database.

Calling random or consecutive phone numbers is essentially a spraying attack. That can be done without having a database of numbers. While they are annoying, they cannot be used to attack a person or a group of people without additional information. That's what I mean in saying that the numbers alone cannot be weaponized.

Australian mobile phone system has a capacity of 100M numbers, which means that with a probability of 1/4 a random phone number is a real one (Australia's population is nearly 26M).

Then why not publish all the phone numbers in clear.

Additional benefits of randomising data by sorting the hashes, for example.

Could you explain those benefits please?

Please accept my genuine apologies here, I think I must have crossed some wires / be operating under different assumptions / be talking past you somehow.

To clearly state my position, I believe:

a) Publishing a list of sha256(.au phone number) is equivalent to publishing the list in clear

b) We are discussing the set of phone numbers of affected Medibank customers, which could be described in a variety of ways (e.g. bitmap, dumb list, hashed list, bloom / cuckoo / xor filter etc).

c) There is basically nothing efficiency wise you need to think about to provide lookups in a database of phone numbers if you don't care about security. It is "laughably small data". All Australian phone numbers will fit comfortably in an Excel spreadsheet, Sqlite database, greppable text file etc.

Finally: Phone numbers and emails are a little weird. They are PII but the seriousness of their disclosure depends on context. Imagine a publicly posted list of porn purchases or medical conditions by phone number. It has a sort of "casual privacy" about it but is quite transparent to each user's contacts.

The data is already in the wild but you do not want to spread it even more, so you do not just publish it. But you still want to enable people to find out if they are in the data set, so you offer a website to check for their data.

You could just keep the data on your server in plain text, even if it would get hacked, there would be no real additional damage as the data is already in the wild. There might however be legal reasons why you can not do this.

So as an exercise in how to handle such data properly or in order to comply with the law, you want a secure solution, i.e. assuming the data is not already in the wild, how do you implement this in a way that getting your server hacked does not leak the data?

The idea was then to just hash everything with SHA-256, but this does not work because SHA-256 is fast and the search space relatively small, so the hashes can be more or less easily reversed. The next better option would be to hash the data with something slow like bcrypt, that would make reversing the hashes orders of magnitude slower with a well chosen work factor.

haveibeenpwned.com also had an article discussing the decisions they made in quite some detail.

You are right; choice of SHA256 is about handling personal data safely.

Unfortunately, the leaked data is available to anyone. There is no point of putting an extra effort of recovering the data from its hashed form.

Thanks all. My "usual go to" answer for this (protecting data where you'd like to hash it but there's not really enough entropy) is in a sibling comment.

Agree, all the data being public already renders everything a bit moot.

not sure if there is an effective solution to this. (phone number specific)

We had to meet a similar requirement and used hardware anchoring.

Instead of a hash, think of a keyed HMAC style operation where the HMAC secret is inside an HSM. HSM operations are handled by a separate team than webapp. Key is not extractable.

It is not vulnerable to brute-force in the same way as a hash because the function cannot be executed outside your computing environment. Of course an attacker can still try to brute-force, but only using the app or from inside your environment.

So you have to apply appropriate rate limiting controls and monitoring on your "boat anchor" HSM for it all to work out.

For this application I would probably look for "proof of ownership" of the number or email before allowing a query.

doesn't "proof of ownership" kill the purpose ? ( my email/phone number not seen by the operator )

My attack scenario was: they don't have the data, harvesting my data.

The way Medibank has handled this breach has been atrocious. Drip feeding information, minimising, misleading etc.

My data is in this breach and all I've gotten thus far is the bullshit comms corporate speak emails from their CEO. No personalised notification that my details are out there on the internet, nothing. There ought to be personal criminal penalties placed on company leadership for this kind of ongoing and, and this point, wilful negligence. Otherwise they will never have sufficient incentive to guard against this kind of risk.

this is kinda wierd...someone in my house has been affected by the leak. So i typed in their name. And it says they're not in the leak. 3 other people in my house including me have apparently all bean leaked according to this site. So, either this resource doesnt work, or medibank is not being forthcoming in a timely manner as to who is being affected. I hope they get sued to oblivion for this.

Does anyone know what one is specifically to do about their home address when it is leaked? It's easy to change emails, mobile numbers and so on...but...home addresses...not so easy.

EDIT: im guessing 1 piece of the puzzle (an address) with all the other things changed should be OK? who knows...

It seems the hackers published only a part of the stolen data.

The same situation here: several people on the same card, some are in the leak and some not.

this is so infuriating, i'm going to change all my details, but to what end? Optus, and Medibank have had leaks...so its a matter of time until this happens again - then what? i change my mobile number/email addresses...again?

The best solution is a single government database of details that companies use to validate the personal data once and with the customers' permission. And this database should be protected better that the PM.

Will the government step up, though? I have some doubts.

this is being discussed now, i think I saw a news story about it yesterday.

This would be great. My fear is that the companies will ask for 5-10 years for migrating to this technology.

We will see. I hope I am wrong.

> Interestingly, .NET 6 implementation of the hash table keeps the items in the (rough) order of addition. To prevent potential correlation attempts, the hashed data is sorted before storing, effectively randomising the order of elements.

Can someone please explain how would correlation attacks be executed in this context?

Reply on news.ycombinator.com