Billion-record stolen Chinese database for sale on breach forum
theregister.com
theregister.com
This is completely unverified though, so take it with a grain of salt.
Static, long lived secrets with limited governance that have no conditional access guards are weapons of mass self destruction.
Refreshing an environment variable that has changed is (for me) a line I won't cross. Time to write the app a different way, once that becomes a concern.
Do you know a way where RBAC can be used for the above?
For us, we're using long lived credentials in this space using IAM Users but with very tightly controlled authorisations.
https://aws.amazon.com/about-aws/whats-new/2022/07/aws-ident...
The oidc subject includes the GitHub org, repo, branch, and environment for the IAM assume role policy to match or filter.
You add the long lived IAM user API key/secret to it and it stores it in a password protected storage (MacOS keychain or similar).
Then you invoke aws-vault with an IAM role and command, and it will handle obtaining short-lived credentials scoped to that role (including TOTP 2-factor code auth), and then run the command with those temporary credentials as env vars.
With the right AWS permissions on your user, it can also automatically rotate the IAM user API keys for you.
Not Invented Here
I still use aws-vault, though, when I'm not in a position to set up AWS SSO.
aws-vault is one of them, though out of support now, aws-okta [1] is another.
Just an FYI it's no longer supported and it looks like the fork has gone stagnant, too.
For example, for AWS you can create long lived credentials for users which are scoped to only allow one operation, namely obtaining a short lived token (with the aid of a hardware token such as a Yubikey) with scope to perform other operations.
AWS guide here: https://aws.amazon.com/blogs/security/enhance-programmatic-a...
> it's senselessly adding fuel to our political division.
This comment, whether you realize it or not, is coming from a place of extreme social privilege.
Remember that for the majority of people, politics is not a game. It is serious. People lose their rights to live the life they want all the time. Sometimes those politics turn violent and people lose everything.
Something got shanghaied isn't a pejorative in the way that Trump acolytes use "China virus".
Are you unaware of the Chinese Exclusion Act of 1882 -- which is exactly around the time that this term was popular and in common use?
At work we codename security issues we are working on for Slack channels, etc. We use unrelated names that you could get from a name generator.
According to this tweet [0] they have a "threat intelligence" department that continually monitors for potential issues. It makes sense that they would be on the lookout for leaks of this nature, as they are highly dependent on correctly verifying and identifying their customers.
[0] https://twitter.com/cz_binance/status/1543700689611792386
I can't count the amount of SO questions I've had to edit from others posting live API Keys for everything from custom services to AWS.
But as mentioned below - Still advised to change your keys for obvious reasons
The only thing you can do is rotating the token/secret.
Something like giving false confidence to the user. Not the best idea.
Of course, then your secret key checker would need to build that string by concatenating so that it wouldn't set off itself.
A project I maintain, Gitleaks, can easily detect "unique" secrets and does a pretty good job at detecting "generic" secrets too. In this case, the generic gitleaks rule would have caught the secrets [1]. You can see the full rule definition here [2] and how the rule is constructed here [3].
[1] https://regex101.com/r/CLg9TK/1
[2] https://github.com/zricethezav/gitleaks/blob/master/config/g...
[3] https://github.com/zricethezav/gitleaks/blob/master/cmd/gene...
Here is a full explanation if you are interested: https://blog.gitguardian.com/why-detecting-generic-credentia...
https://res.cloudinary.com/da8kiytlc/image/upload/v164614852...
[1] https://regex101.com/r/CLg9TK/1
[2] https://github.com/zricethezav/gitleaks/blob/master/config/g...
Is it covered by a different rule perhaps?
> but I assume they were chosen based on the statistics?
Nope, not statistics. Identifiers and keywords are chosen based on what I see out in the wild being a software engineer.
Use vault, env vars, GitHub/GitLab secrets, anything but string literals!!!
It’s end-to-end encrypted, cloud or self-hosted, and very quick to integrate.
Is there a plugin that streamers could use to blur suspected keys on stream? Would that be something interesting to work on do you think? (I'm not a streamer but it sounds fun)
My main precaution though was separating dev/prod and never looking at prod stuff online. Worst case someone could spin up some guff in my dev/test account until I can cycle the credentials
In my case the separation also included a different system user on my computer for stream work. Possibly overkill but why risk it when the costs are so low?
I can't see myself trusting a key blurring app if I'm honest. Rather fix the issue earlier in the process than rely on something that would probably break on edge cases (word wrap enabled? Here's the key but it's in two parts, that sort of thing)
https://twitter.com/_KarenHao/status/1543949945614393344 (thread)
Basically just about every app (YouTube, Reddit, Facebook, ...) is better this way. I.e., no ads, erase-able elements, less spyware, defaults to no notification and sometimes even gets better functionality. For instance, it (browsers) gets rid of "hearts" in Duolingo for whatever damn reason, so you can practice however much you'd like in a day.
The downsides I've found is that you seemingly can't Chrome-cast from it, and it often creates new tabs instead of reusing existing ones or making it's own app-instance, so you gotta close all tabs every so often.
Hacker claims they stole police data on a billion Chinese citizens - https://news.ycombinator.com/item?id=31984663 - July 2022 (1 comment)
Hacker claims to have obtained data on 1B Chinese citizens - https://news.ycombinator.com/item?id=31980101 - July 2022 (1 comment)
Hacker claims to have stolen 1 bln records of Chinese citizens from police - https://news.ycombinator.com/item?id=31977354 - July 2022 (1 comment)
Police data of 1B Chinese people leaked - https://news.ycombinator.com/item?id=31969617 - July 2022 (4 comments)
Shanghai Police leaking 20TB Chinese citizens data? - https://news.ycombinator.com/item?id=31962526 - July 2022 (3 comments)
It sucks when you're earlier and don't 'win', but it evens out in the long run if you post lots of good stories, since sometimes the lottery works in your favor. One of these years we'll get around to implementing karma-sharing to spread credit across multiple submitters.
This title is not exactly correct, yes the data involves lots of police reports, but it may come from a thirdparty, as Chinese police would often outsource data analysis job
It seems the majority of people on the planet now have had some of their data leaked. Or are becoming ever more entangled with government and corporate systems which control and peddle their information as they see fit.
Is it ultimately a big nothing burger, or is this some singularity we are passing through?
Obvious comparisons to e.g. the Netherlands' famous over-registering of religion and how the Nazis abused that. But I feel this is long term potentially worse than that. Not in the level of horribleness, but in the effect on society moving forward.
In Denmark, every child was (I’m not sure if they still are actually?) registered at birth by the local parish in so called “church books”.
With these “databases” in hand, the SS had a neat list of all names, and the approximate location of peoples homes.
Those lists were used to identify and prosecute jews.
"Machine-tabulated census data greatly expanded the estimated number of Jews in Germany by identifying individuals with only one or a few Jewish ancestors. Previous estimates of 400,000 to 600,000 were abandoned for a new estimate of 2 million Jews."
[0]: https://en.wikipedia.org/wiki/IBM_and_the_Holocaust
The IBM subsidiary in Nazi Germany selling and maintaining the tabulating machines was DeHoMag, Deutsche Hollerith Maschinen AG.
...
If you know who to rule out, you have a smaller pool of people to go after.
That was the price, the defeat of their last hope against the Allies. All of the Great Jews that slapped those firecrackers together were exiled due to antisemitism: Fermi, Szílard, Einstein (to get the president to read the letter to get the Los Alamos show on the road in the first place, get Roosevelt to read top to bottom left to right, no easy task), von Neumann (spesh because of his schizophrenia, no concentration camp for him, he would have been experimented on to then do that same sin to everybody in the camps, Schizophrenic Jews were at the absolute bottom o the Nazi world order).
I just posted about this. https://news.ycombinator.com/item?id=31990431
Fermi was originally a fascist, it basically made sense to him as a way of organizing a country.
Only non-Jew in the top desks of Los Alamos. Why? Only when the racial laws against his Jewish wife and children did he pack his shit and leave for America.
And Fermi was packing heat.
Hans Bethe James Franck Edward Teller Rudolf Peierls Klaus Fuchs Otto Loewi Max Bergmann Dieter Gruen Lilli Hornig
I also forgot many in this list.
Same in Sweden.
-----
Tangent: the info pages on the Anne Frank House site have sections cycling through different pastel background colours.[0] I've wondered before whether something like that would the brain acquire context in a long page, making comprehension more like that of a physical book. Seeing it implemented, it doesn't seem to help. I think being able to easily flip to a previous page and back was one of the advantages of printed paper, so maybe a sticky TOC with the same colours or a minimap scrollbar would allow that? Actually, why not have that standard in browsers?
Hmm, the concept of coloured sections was known in 2013 already.[1]
[0] https://www.annefrank.org/en/anne-frank/go-in-depth/netherla...
[1] https://ux.stackexchange.com/questions/62808/website-layout-...
> I think I first read about it on HN, actually
That may have been my article:
I actually saw the fact pointed out in a comment. It's brought up quite often here—even a fairly narrow query finds many instances:
https://hn.algolia.com/?query=netherlands%20religion%20nazi&...
Some have citations, too. HN is proving quite useful as a knowledge engine.
In fact, information in the government's hands is the most dangerous, because they have more power than anyone else to use it against you.
(On the other hand, as others have said about Denmark and Netherlands, data that was not in government hands became in government hands, and was used against people. So it's not "safer" if it's in private hands, except to the degree that the government has to go through the extra step of getting it.)
Also they keep the record of the delete request, which contains the PII you ask to remove.
At this point I've basically accepted that all my info will be found on sites like fastpeoplesearch.com and that anything I tell any company (or I guess in this case, govt too) will eventually be leaked, correlated, and used against me.
names and relations to family members and all their phones and addresses
previous addresses
a lot of it is collected from voter registration data (so your party affilition can be gathered as well)
I was royally pissed when I moved into a new home and literally a day after I signed up for internet service with Spectrum cable I got spam calls that know what state I'm in and my new home address is up online before I ever get around to updating my ID etc so I assume my data was sold immediately by them
Well, if you look at (global) society as a dynamical system it seems to me that there are two stable basins or attractors, call them "Star Trek" and "North Korea".
In the "Star Trek" future the people in charge are themselves also subject to the panopticon, and the world is ruled fairly and humanely. (The other name I use for this is the "Tyranny of Mrs. Grundy".)
In the "North Korea" future there are (human or AI or hybrid) masters and brain-chipped cyborg slaves, and rule is absolute and enforced with digital precision.
(Of course, this is all predicated on the idea that we can't put the genie back in the bottle in re: ubiquitous surveillance. I think that's likely the case (although I do not like it) but I'm not going to make the argument here unless someone asks.)
Given the above the thing to do is work to make politicians subject to 24/7 total surveillance (ASAP, before everybody else) so we can keep an eye on them. This policy would also presumably weed out the crazies and corrupt, eh?
Nice analogy. Do you really believe, that us being on an utopian trajectory is realistic?
Oh yes. Very much so. In fact, by many measures we have been on an utopian trajectory for several centuries. Today even our failures are the result of unimaginable power. We have to learn to wield our power with wisdom.
We have all of the physical technology we need already. We can practice regenerative agriculture that increases topsoil fertility and volume; we have methods of construction that can build housing for everyone; machines and factories that churn out the physical necessities of life; etc. We need only deploy our resources and technology efficiently. It's down to logistics now, and we have more than enough computer power to sort that out.
The only thing holding us back is that most people still don't realize this yet.
https://en.wikipedia.org/wiki/Design_science_revolution
It's taking longer than anticipated but the process never stopped. Reading this now you are a part of it, if you want to be.
I have a mailing list: https://lists.sr.ht/~sforman/heliotrope.pajamas
And I lurk on (my own) IRC channel: irc://irc.libera.chat/#MagnusMotive
I'm pretty weird and flakey, but I'm also committed and serious (though not humorless) about working towards the nice future. :)
I see you have an email addy in your profile, I'll email you a little later today, after more coffee.
The standard "leak" of names and addresses of people is totally meaningless, though HN "privacy" obsessives blow it out of the water all the time. It's basically public information, we used to have everyone in phone books in the US and almost no one cared.
Cell phone number is a riskier one because of the opportunity for 2FA hacks. It's not hard to get people's cell phone numbers as it is (you can buy direct marketing lists for pennies per person in the US) but its not good to make it easy for hackers.
However this leak in particular appears to go much deeper so it is insidious. Police records are named and who knows what else. That is a genuine privacy issue and sucks for those involved.
You've chosen some arbitrary amount of information where you begin to care and become interested, and decided everyone with a different cutoff is an absolutist you don't need to listen to. But it's really just that your situation permits you to leak that information without fear, and you haven't deigned to imagine that other people are in a different situation.
I'd encourage you to rethink this perspective.
I was thinking - if I had this, what could I do with the personal records of a billion Chinese people?
And I must conclude - absolutely nothing. It's of no interest to me.
Now, I probably lack sufficient criminal imagination, but the point is stuff like this is hard to fence because there's a very small market of buyers. In an article I wrote for Routledge about the markets for stolen digital data (specifically movie and album releases) I suggested that the underlying problem is there's symbiosis between leakers and buyers.
If you want to do anything, target the buyers. There's less of them. Don't try to secure inherently insecure massively centralised systems (Blotto + Dolev Yeo problem) . Or chase leakers. Or blame users. Or fire the CIO. Find out who wants this stuff and take down the show from the demand-side.
But hold on! Guess who the buyers are. And guess what sincere will exists within "law enforcement" to tackle this sort of "cybercrime".
Having a definitive record of people's existence would make it more difficult for the authorities to skimp on natural disaster rescue efforts then lie about casualty numbers, treat citizens as canon fodder for military purposes, or simply wipe out individuals who have grievances with the government or powerful functionaries.
It’s also useful for more targeted social engineering attacks.
And I'm sure that plenty of gullible people were scammed and lost their money because of those leaks. When someone calls you, knows your full name and talks with enough confidence, it causes some trust.
It’s a little lengthy, but it’s cut down on the number of spam calls I actually answer specifically and I’m reasonably sure that anyone who actually needs to get a hold of me has an easy path to do so.
you use personal details to tailor phishing scams to the victim
If I know you have a grandson that lives in XinJiang I can cold call you and say i'm from the xinjiang police and we have your grandson here under arrest and then extort money from there
Stop using personal information for authentication and end the concept of “identity theft”, for starters (not sure how it works in China specifically).
Include online safety and anti-scam curriculum in lower education.
There was an HN post about this a few months ago:
https://news.ycombinator.com/item?id=29654137
Someone posted a comment explaining a little more about Shanghai's special relationship with the CCP/PLA:
>Shanghai is a city with a unique role in the progression of the CCP and its global efforts. Also PLA Unit 61398 is in Pudong, the shanghai district mentioned in the article. Overall there's a lot of CCP/PLA-adjacent tech talent in the area, and of course the local police still ultimately report to the CCP.
Why? Tons of Software was written for XP, and then abandoned without any support. Many of that stuff in the government sector. A lot of online banking clients outright say "only works on XP," and copyright years reads 2006.
This is similar how Android 7+ support was almost nuked in China for nearly a year because Tencent didn't want to port Wechat to newer APIs cuz "nobody uses Android newer than 4.X in China"
Access just based on credentials seems so wrong anyway. There should always be whitelisted IPs for sensitive stuff like that.
The Shanghai police thinks like you, so they purchased a very expensive "private deployment of Alibaba Cloud", which in China usually works like this:
1. The customer build a data center.
2. Alibaba Cloud purchases servers, deploys them in the customer's data center along with all Alibaba Cloud software (same as in the public cloud).
3. Customers do whatever they want to the thing.
Basically by "private cloud" they really mean it, something AWS won't ever do.
In this case, the system is technically "not connected to the Internet", but we all know what this mean: it certainly will be occasionally.
Most cases I know, the customer cite "data security" as the reason why they would like to do this, because on-prem are always more secure right? But I hope we could agree on why this does not work:
- It is now very difficult for Alibaba Cloud to do ops work on these private deployments, so ... there will be maybe 2 releases per year, or in some cases never, including security patches. It's not rare to find a 5-years-old struts2 vuln in the control plane of such private deployments, and in the coming years it would be log4j2 I guess.
- Alibaba Cloud put serious effort into securing their public cloud, and even covering the ass for the customer. For example similar to GitHub+AWS secret scanning, they also proactively revoke access keys once the key appears on the Internet. The customers, on the other hand, usually do none of these.
In short, security is largely an Ops work and economies of scale also work here.
In the end these on-prem systems depend solely on network isolation for their security, and... air-gap does not always work.
How could anyone possibly make money off this data set?
I could understand if the Chinese government would pay for it to avoid embarrassment but making the sale public kinda voids that.
Foreign intelligence agencies for classic espionage. If you want to do blackmailing in china, such a DB would be a good start.
Otherwise, data brokers. Advertisement, financial credibility, trustworthines of buisness partners etc.
As for "data brokers. Advertisement, financial credibility, trustworthines of buisness partners etc.". Maybe. But these companies would turn themselves into criminals by using or purchasing this information.
"But these companies would turn themselves into criminals by using or purchasing this information."
Which is why they probably would not deal with the information gathering directly, but use a service of a data analyst company. When they do something illegal, nobody who contracted then did ever know anything. I think this game is played in china as well.
this data is only interesting to the low end of data brokers, advertisers and other scammers, hence the rather low price.
What can you say publicly?
some guys at the top of the game are probably already doing this and have figured out how to both insulate themselves and launder/hide data they horde.
Someone/some team in the police department is probably in serious panic right now. Not only because the data is leaked, but also because the leak has displayed an example of what they are actually recording.
For example, according to the posts that other people has posted online (probably rumor and speculations), the `address_merge_with_mobile_data.json` file is a collection of external data submitted to the police database. In the file, there are data source types such as:
- shga_dwd.base_shangyun_lhrytbxx_df
- shga_dwd.base_wahlw_base_teladsllibrarytab_df
- shga_wa.ods_nb_tab_goods
- shga_wa.ods_nb_app_icpoof_expressdelivery
- shga_wa.ods_nb_app_icpoof_delivery
- shga_wa.ods_nb_app_icpoof_expressdelivery
- shga_wa.ods_nb_app_icpoof_foodorder
I guess if you're ordering food online in China, probably need to give yourself a nice nickname first instead of just using your real name then.
Is that a … concept in China? That the police should only have data “ directly related to census, social safety, or law enforcement”?
I wonder how strange or surprising to locals this might be or maybe not be?
The altitude of most people here (I'm a Chinese BTW) is "I did nothing wrong, so why should I care?".
But, it's one thing to hear the humor, it's another thing when you can actually see it. People will have different opinions about privacy when their wives asks them why their business trip to another city showed up as a hotel night 500 meters away from home on the government database.
Had a good chuckle. But then it got me thinking about other situations in which government-leak-induced friction that might lead to cause for legal action. The government demands and collates data presumably supported by legal under-pinnings that citizens must comply with. When these are leaked, shouldn't the government bear legal, fiduciary, etc. responsibility?
With great data, comes great responsibility, no?
The law has also stated that if an institution violated the law, and civil damages is generated as result, the institution is also responsible for the damage.
There is another law which might also apply to this case, it's called Data Security/Safety Law. This law also stated that the institution is responsible for civil damages if the institution has violated the law.
But, I'm not a lawyer and certainly not a Chinese lawyer. This situation is fairly complex already, it's hard for me to tell which direction this case would go.
I guess they'll investigate it first, but the company/team which worked on the project has probably already read the laws before, thus I assume they're well-prepared for this. Another twist is the programmer who accidentally published the secret database API key to the public network, this needs to be investigated as well to determine whether or not the key has actually been used in the attack.
So I guess in the meantime, the husbands must wait.
https://www.wired.com/story/billion-records-exposed-online/
Appears this leak is a single dataset — one I linked to is multiple datasets.
Some example news: https://www.privacyaffairs.com/facebook-data-sold-on-hacker-...
- 750k row of sample data is large enough for a leak by itself, many on reddit/twitter/fediverse have already started to explore the data set for gender ratio, age composition and frequency of raping cases, etc.
Any links?
https://www.reddit.com/r/China_irl/comments/vr214w/
https://www.reddit.com/r/China_irl/comments/vr2lij/
https://www.reddit.com/r/China_irl/comments/vqfwic/
Take these threads with giant grain of salt though, they're far from thorough and some of them lack basic understanding of statistics. And I personally don't think the dataset (at least the sample) is actually random so not really a good representation of China's demographics.
>many on reddit/twitter/fediverse have already started to explore the data set for gender ratio, age composition and frequency of raping cases, etc.
Is a portion of it public?
Isn't BTC more traceable than Privacy coins like monero?
how many billions?
One gets used to short scale on the Internet.
Personally I don't expect this to bear true. Historically in China, government failures have been cited as evidence for further centralizing the power of the federal government. And this argument is bought hook-line-and-sinker by the people. I don't think that will change until there is serious economic hardship.
just look at north korea and cuba if you want to get a sense for how long these regimes last. USSR was an exception.