China’s plan to organize its society relies on ‘big data’ to rate everyone
washingtonpost.com
washingtonpost.com
The IRS rates you on likelihood of fraud, border patrol rates you on your likelihood of breaking residency law, the FBI rates everyone's chances of being a terrorist, credit agencies rate your worthiness of a loan, insurance companies rate your chances of being a cost, banks rate your likelihood of paying a mortgage, your friends rate your worth and you let them, and if you're in the media biz everything comes down to a numbers game. Your income is used to sort you into pidgeonholes in virtually every aspect of your interaction with every social institution.
Your worth as a human is already determined by a set of numbers; we've just done it so much better than China, because nobody noticed.
* The financial crisis was partially a result of mortgages being given to people whose financial data would have suggested that they had no business borrowing so much money. In that case, the opposite of your point is true. This shows that attention to data is beneficial to a mortgage system because it helps avoid questionable loans.
* My friends don't rate my worth, because I don't have friends.
Were credit score non existing in 2008? If so, were they completely ignored?
Also, in comparison, European countries with non existing credit scores didn't have a crisis with subprime mortgages, probably because they paid attention case-by-case to the loan applicants. They surely used data, but it does not mean that this data has to be centralized and behind black box algorithms.
I think the parent's point may be incorrect in this example. The financial crisis didn't happen because the lenders weren't aware the clients had a high likelihood of default, it was because the lenders didn't care the clients had a high likelihood of default.
The mortgages were packed into packages and sold to other entities so the lenders who initiated the loans passed along the risk. They knew very well that the clients were high risk, but they were incentivized to ignore those risk factors.
And yet, he was spouting complete rubbish; all the assertions that he made were contrary to straightforward treatments of the data and common sense. I can't even remember the technical justifications he made, I think that I probably didn't make appropriate challenges to get these.
So, I went back to my job as the telco doing pedestrian things with telematary data (n things not working = problem). The lesson of the conversation is that the smart people who were figuring out risk and investment strategies in these places are not smart. Also I was not of the caliber required to smoke that out.
An actionable lesson that I have run with since (ok, I thought this out in a slow way about 18mths after the crisis while chewing over past humiliations and social disasters in which I have participated), is that critical thinking is the most important skill in my portfolio, and I'm not good at it - and almost everyone I meet is worse than me. So I have focused on trying to recruit people who seem capable of thinking round problems, and then have invested time and energy in getting them to be able to understand problems and become domain experts. This is different from the normal strategy of recruiting experts, doesn't deliver results reliably (which is challenging in a business environment - I have missed my scorecard a few times since this experiment began and I am rendered nervous about my career as a result) but has produced a business capability which to me seems qualitatively different from the previous capability that I took over when I got my current role.
Anyway - it makes for more fun at work.
Lenders would make money by originating loans and selling them to larger financial institutions, so they had no skin in the game when it came to counterparty risk. Lenders certainly had access to FICO scores, but they had a profit incentive to pitch and approve loans to unsuitable borrowers, and there is a strong argument to be made that they used passable FICO scores to justify bad loans to people with no income and no assets.
https://en.m.wikipedia.org/wiki/Predatory_lending https://en.m.wikipedia.org/wiki/Subprime_lending https://en.m.wikipedia.org/wiki/NINJA_loan
Separately, one of the many differences between Europe and the US leading up to the crisis was that thanks to years of comments from Alan Greenspan, it became culturally acceptable and even encouraged for American borrowers to "tap into their home equity" by taking second mortgages. So, when houses got cheaper, many people who had been paying loans for a while still owed plenty of principal. One might argue that there was negligence on the part of second mortgage lenders here.
The lesson is that the risks of finance need to be sufficiently crystalised that behaviour is moderated, the problem is that economic growth has come to depend on risky finance.
Deliberately, no. But they knew it was unsustainable. Many industry insiders, as well as outside observers, likened the system to a house of cards. And when you play with a house of cards, you know damn well that it will come crashing down sooner or later.
Those ratings were created as a result of significant conflict of interest. You should read John Bogle's treatise on the subject (he's the founder and retired CEO of The Vanguard Group):
http://johncbogle.com/wordpress/wp-content/uploads/2006/02/P...
"Market participants—now dominated by speculators, not investors—also joined the parade of miscreants, and our professional security analysts failed to do their job of appraising company balance sheets, largely ignoring the huge credit risks assumed by the new breed of bankers and investment bankers. And let’s not forget our credit rating agencies, which happily bestowed AAA ratings on securitized loans in return for enormous fees that were paid in return by the issuers themselves. (It’s called “conflict of interest.”) Yes, there’s plenty of blame to passaround."
I think it is worth noting the subtleties around "they", though. Some people thought it was unsustainable. Others were dumb and didn't. Many didn't care and just wanted to be paid, now.
Sorry to hear that.
More to the point, your potential employers don't get access to your personal life, beyond what you divulge to them and whatever information they get from public records (sex offender lists, states with public arrest records, &c). They don't know how "good" of a citizen you are, your mental health record, or if you've ever gotten a speeding ticket. China's proposed system opens all of that.
The US may have many real privacy violations but you have at least an ostensible right to privacy.
That authority figures don't have to pretend fair even if they aren't actually fair is worth something. That surveillance isn't official is worth something. Sure, solid guarantees to privacy would be better but one really shouldn't claim unofficial surveillance is the same as official surveillance since the move from one to the other needs to be resisted also.
- people are tracked by everyone. Companies. State agencies. Other people.
- the database contains much more data than you think it does. And there are more data base than you think, also most of them are not connected to each others. But people in charge are.
- the data can be exploited in much more ways than you think it can.
- the law is broken is many more ways than you think it is. And as often as it's possible when not getting caught.
- the human in charge bypass everything they can, with every way they have, and help each others to do saw across entities.
- even the human that are not willing to do any of these are not necessarily competent enough to prevent abuses.
The only we don't suffer too much about it is:
- you can't see the many things in your life that are influenced by it because it's indirect and untold. - the sheer mass of it make it hard for the human to handle everything so only 5% of the real potential is actually in used. Wait for the AI to step in... - those are life threatening most of the time. But remember the table of power and freedom always turned during the history of humanity. Our current situation is NOT permanent.
Luckily most people I've met related to the data collection are not evil enough, and don't have enough greed to be a threat. Even the ones that are are usually not competent enough to become the next big brother. But the potential is here nevertheless.
TL;DR: the amount of data is huge. It can be used in scary ways. It is abused on a daily basis, but the consequences are not too insufferable yet.
I've thought about this issue a lot (I'm a data-herding obsessed guy myself), and I think that this is one of our last chances at decent privacy, I mean, the fact that there are still many unconnected databases containing our data instead of one big blob where search for relevant info would be much easier. I'm still hoping that enough things get lost in "translation" between these several databases so that the info becomes less intrusive, and that some of the databases will eventually become "deprecated" i.e. data will become stalled.
> It is abused on a daily basis, but the consequences are not too insufferable yet.
Just the other day I found out about a guy whose credit application had been rejected by the bank because he had gotten a police ticket a couple of years ago for "public disturbance" (some neighbors called the cops when he was throwing a party at his house). What was crazy is that the matter hadn't gone through any legal court, it was just saved as a ticket stored in the police's database and probably the local tax-office's DB (the police tickets are collected by the local tax office). I can't really fathom how a private entity (the bank) got hold of that information. (I live in Eastern Europe, if it matters)
Is the fact that private companies do the tracking supposed to make me feel better? It doesn't; it only increases the likelihood that the information will be sold to unknown 4th parties or auctioned at bankruptcy fire-sales.
Not even a month has gone by since the story broke of law enforcement buying data off Twitter's firehose from a private company to track protesters. See also: parallel reconstruction.
YES!! A government has much more power than a company operating under it, particularly in a single-party state.
for a study of north american bureaucracy - 'The Utopia of rules’ by David Graber (my summary here http://mosermichael.github.io/cstuff/all/ramblings/2015/03/2... )
for the study of Russian/Soviet bureaucracy - 'Dead Souls' by Nikolai Gogol http://robobees.seas.harvard.edu/files/gov2126/files/gogol_d...
Don't you think it can be a big threat to national security. And why does Chinese government do that?
Putting a number on someone isn't inherently evil. The old system had numbers too, they were just in people's heads instead of in a computer. And as I said, they were much less accurate and much more biased.
China is trying to punish people with different political opinions or behaviors. The american system doesn't care what your political opinions are, just that you don't commit tax fraud and pay back your loans on time.
I.e., it is mainly for commercial/security reasons in the Western world, not for political control of the population.
Should we turn credit into a market for lemons? Just ratchet everyone's access to credit down to what's now granted to those with bad credit? Go back to the days of "how respectable do you look walking into the bank?"
Should the IRS spend its finite resources looking for fraud where it's less likely to be?
Should middle-aged people with clean records in Accords pay the same premiums as people 19-year-olds with DUI convictions in WRXes?
Should the state assign friends/spouses by RNG and enforce that you spend equal time with each?
People are different. They have different personalities, different behaviors, different risk profiles. It makes no sense to ignore these things. In the organization examples you've given, at least, it's much better to use data-driven, evidence-based models kicking out scores than to rely on the prejudices and informal gossip networks of gatekeepers - like we did before we were sophisticated enough for records.
Even where scores aren't appropriate, you're going to allocate your resources - like attention - to people you enjoy over people you don't.
China's system is problematic because it's assessing something we think should not be assessed or considered anywhere - loyalty to the Party. But they were likely already doing that anyway, with informal gossip networks among Party officials. Executed correctly, the "social credit score" system should serve that end more accurately, with less corruption and ultimately more egalitarianism. We just don't think the end should be served.
Though I think it was that one that's based on a Charlie Brooker story, rather than written by.
I actually paused after to decide if I was going to bother with the rest at all. 2nd episode (as far as I've got) was much better.
Watching Nosedive, I thought how close to it we were already, it's just that we're measuring people in terms of Followers or Likes. This path started I think when we prioritised Google Analytics numbers over the positive effects you create within your audience: clicks, traffic, unique visitors all seem to count more in 2016 than loyalty and shared values that dominated publishing for the prior 100 years or more.
In the age of "personal brand" it makes sense to me that we're going to end up with "personal metrics" and we're just seeing the tip of the iceberg.
At the end of Nosedive I decided if that was the future of my society, I'd stay being me and suffer the consequences. I'd rather be happy in my own skin that be popular, and if I lived in a society that turned popularity into a measure of personal worth, I'd quite happy to be a dissident within it.
It just felt too "plasticky", that conclusions about the worlds displayed were being forced on me, instead of giving some room for thought.
Not trying to start an argument over matters of taste, just interesting to see how opinions differ! It does seem like in season 3 -- aside from the episode everyone is talking about in relevant to this thread -- they run with more of the "sci fi premise" approach, as opposed to making you feel like this is a just around the corner, as some of the first seasons did.
But determining business risk is different from determining whether someone is a moral or immoral person, and then using that subjectivity to control the population. It's value to the regime is in that subjectivity: If you threaten my political aspirations, I'll score you as immoral, report your transgressions on the nightly news and toss you in prison to be reprogrammed.
The US separates church and state to prevent this.
* I freely admit to being biased on this issue, but the fetus has a heart-beat at 3 weeks; it's hard to say it's not living being. And if it's living, aborting/terminating it is technically killing it.
The duty of the judiciary is to interpret and apply the law. The creation of the law is the job of the legislative branch. It seems to me that while people may agree or object to laws on moral grounds, the judicial branch is focussed on the application of the law itself, not the moral aspects of it. As the legislature creates the law, one could argue that they're more directly responsible for encoding morality into the law. Though some might think in some cases the judicial branch abuses it's interpretive duty by "legislating from the bench".
What do you think? Edit: I suspect we're actually quite close to agreeing.
I think what upsets people about "legislating from the bench" is things like at least 25 states pass laws or constitutional amendments banning some form of homosexual marriage [1], which clearly reflects a substantial will-of-the-people, and the Supreme Court overrules the expressed will of the people and says it's unconstitutional. So now these people, have to live in a society that legally tolerates something they think is wrong and/or harmful to society long-term. Regardless of your view of whether the Court was "right," you can certainly see why people would be upset with this. No matter what the Court rules, someone is going to be unhappy, but given that homosexuals are 3% of the population and are driving the other 97% (which had expressed their desires through the democratic process), you can see the problem.
(I think this is why conservative Christians are voting for Trump in large numbers, despite the fact that many of them think he is not a good candidate. They are voting for a Supreme Court nominator, not a government leader. As a conservative Christian myself, I think this is foolish, but there it is.)
[1] https://en.m.wikipedia.org/wiki/List_of_U.S._state_constitut...
But either way, the percentage of people who believe certain morals does not matter. No matter what our individual beliefs are, we bind ourselves to the law of the land. That's what it means to have a constitution. Even if 100% of Americans wanted to take away your freedom of speech or your freedom to bear arms, we could not do it legally.
In this discussion I'm not taking sides, just noting that the court needs to decide quasi-moral questions. Our interpretation of the Constitution depends on our current beliefs (and might even lead to an interpretation that the original founders would have rejected). The constitution clearly says that we have a right of free speech. However, as far as I am aware, it does not state that we have a) a right to marry, b) a right to do homosexual acts, or c) a right for homosexuals to marry. In fact, I believe B and C were illegal for large periods of time (anti-sodomy laws). I suggest that the Founders probably would not have seen homosexual marriage as a right. Due to changing morals, however, the Supreme Court now views it as a right. Were the laws constitutional before but not now? Were they always unconstitutional, but nobody challenged them? As a thought experiment, if societal morals changes to believe that homosexuality is actively harmful to society, would anti-sodomy laws be constitutional? The process of interpretation of rights necessarily involves our current beliefs.
In Roe, the court found that outlawing abortion would constitute a violation of rights already established in the Constitution, specifically the due process clause of the 14th amendment. This means that the same legal rationale for your right to abort is also the legal rationale that prevents a state from, say, passing a bill stating that "prewett is now a criminal because we say so."
In the case of Obergefell, the court decided that denying recognition for same-sex marriages from other states was a violation of equal protection, because opposite-sex marriages from other states were recognized. This is the same right that, say, allows you to marry someone of another race and then go to another state and have that marriage continue to be valid.
So no, the court is not deciding, morally, what is right and wrong. If you think that, that's probably the source of your confusion. The court is deciding that the implication of the constitution and laws as written demand this conclusion in order to be logically consistent.
You might disagree with the logical (or moral) basis of their decision. Sure.
You might say they're legislating morality. No.
Something really bad is happening in China. There's been talk for a few years about abolishing the hukou system of housing registration, which would appear to be a step towards liberalization, but at the same time the government seems to be running full tilt in the other direction, developing new systems for control. I just left China after fifteen years, and when I moved out of my apartment in central Beijing, an Indian guy was considering moving in after me. The local police station told my landlord, "no Indians". Later they told her, "no one without a Beijing hukou can live there", which is pretty staggering if true, as the majority of Beijing residents don't have Beijing hukou.
I don't know what's going on, but levels of paranoia in the higher reaches of power are spiking, big time.
Well said. This can be said about pretty much any "system" being introduced in China.
Your scientists were so preoccupied with whether or not they could, they didn’t stop to think if they should.
If the Chinese government is viewed as one giant organization then again this is just a logical extension of such systems.
They are just taking a cue from credit rating agencies.
People's access to credit should be affected by how they behaved with previous loans.
> The FBI and CIA I think also collect statistics and profile relevant individuals.
Citation needed (especially that they do it for everyone).
1. We know that they can. If you were designing a mass survaillance system, you would store all incoming information into unique bins now, then later select on the bins of interest.
2. Three years ago I asked an ex-govt employee who would know. When I got to that exact question, he got visibly upset and abruptly ended the conversation.
You know what might make more sense? The Chinese govt. (or any other similar aspiring govt.) should spend billions on BCI research and implant chips in everyone so that any dis-harmonious thoughts can be culled right at the source.
If this model really works for China and other non democracies, so they can keep a liberal economy and still control people hard politically, then:
1984, here we come. :-(
If banks or insurance companies can figure out how credit score changes of Facebook friends predicts you financial status, better abandon them if they screw things up.
https://mic.com/articles/123452/facebook-to-let-banks-discri...
"Beer was a leading theorist of cybernetics—a discipline born of midcentury efforts to understand the role of communication in controlling social, biological, and technical systems. Chile’s government had a lot to control: Allende, who took office in November of 1970, had swiftly nationalized the country’s key industries, and he promised “worker participation” in the planning process. Beer’s mission was to deliver a hypermodern information system that would make this possible, and so bring socialism into the computer age."
So "It's a company, not a government" is not a useful argument to make for this kind of situation.
The Chinese "social score" system would almost certainly be developed and administered by private companies (prototypes and precursors have been). The government would just set the rules. So in fact it would be similar to the US credit system in this way, one system, dictated by the government, private entities just handle its operation.
But companies like Airbnb have to use proxy metrics like number of Facebook friends. If I could be proven to be the trustworthy person I am, then a host wouldn't need to look at all that. If I'm known to have destroyed someone's home, then someone else on Turo knows better than to rent to me.
In practice, it's likely to turn into an eBay style "A++++ best person ever. Let me on the bus brilliantly! 5/5" but the idea isn't terrible on its own.
Follow up question: would this be good because it was by the US government?
This may have changed since when I signed up years ago.
It's just like your GitHub is a reputation aid when applying for a job. It's not required but what you have on there can help.
EDIT: In case it isn't clear, I'm not claiming here that Airbnb is forming some sort of transcontinental people database.
Imagine a world where saying something bad about a company like Airbnb means you won't get a loan from the bank because the owner is a communist party member and your worthiness score is now reduced to an unacceptable level.
This is a bad idea not because it's the Chinese government. The Chinese government is bad and this is just one more terrible idea that shows us just how far they are willing to go to control the population in perpetuity.
Western critiques of China and Russia are directed at the regime, but if the regime collapses, there is no on to criticize, but people still suffer. That is, the Western critique is not fully utilitarian: it criticizes the regime for taking measures to ensure stability, without considering the consequences of the regime losing control.
For reference, about 70 million Chinese died after the collapse of the Qing dynasty (caused in part by Western intervention, by the Opium wars), and Russian life expectancy reduced significantly during the power vacuum between the fall of communism and the rise of Putin.
This gives some background on why both countries support their governments: they want stability and strong central government because they know the alternative.
If Westerners want to adopt a moral and convincing approach to human rights in China and Russia, they need to explain how these things can exist without causing a collapse of law and order that would be worse for the ordinary citizen. This would require reigning in the CIA so that countries could loosen the reigns without fear of a color revolution or Arab spring.
What country is he talking about?
America has the credit rating system. With much hand wringing and energy devoted to increasing or altering the score and so on (companies advertising a way to increase it for you, landlords claiming they'll ruin your credit rating if you ask for a deposit refund ...).
Now imagine if the system was even more corrupt and you could pay someone under the table to alter your score directly. Now imagine that is not just a number for your credit worthiness, but a number for your, well general-life-worthiness. There is no way that is not getting exploited, hacked into or gamed. Maybe that is the ultimate goal, it lets those who control this system, control the population?
Imagine an implied threat of getting -150 points for joining a protest. Or maybe your relative is going to court, you imply to the judge he might get a +50 if he lets your nephew walk. Possibilities are endless, as they say.
Now chastened its more of legalese and tiny triumps based on hope rather than scrutiny.
Just as well the sordid culture of using human rights to score moral points and signal cultural superiority is done and the remaining vestiges of apologism and denial should think seriously about self examination and scrutiny lest the ground shifts even further under their feet.
Basically they're driving out those who don't have Beijing housing registration. The propaganda is addressed to those who will enjoy the relief after the undesirables are gone. No one speaks to the undesirables.
The one-child policy may be over, but the Chinese government's manipulation of the lives of its citizens is alive and well.
[1]: https://mp.weixin.qq.com/s?__biz=MzA3Mzc3MTc0Mg==&mid=265293...
Edit: Here's another link (sorry, also in Chinese) from June, laying out the bones of the policy to drive people out of the capital. Again, posed as a great policy with no downsides. http://www.china.com.cn/guoqing/2016-06/17/content_38687237....
Communism/Marxism is one of the worst ideologies out there (along with intolerant religions). The western democracy is the best model the humans have invented yet. This is not to say we must not strive to improve it. But Communism/Marxism is not at all an improvement, it's an outright societal cancer. But I am seeing is the increase in criticism of US (and western democracies) by leftists and liberals. That in itself would not have been a big deal but these leftists praise Communism/Marxism.
This is a wake-up call to the people of united states who are listening too much to the unjust and undue criticism of the US govt for surveillance. Post-Snowden some leftists are taking undue advantage of the atmosphere to spread their fear mongering anti-democracy propaganda amongst US citizens.
That's your opinion. FYI, China has the dreaded communist politburo running the show.
China is not at all capitalist as there is no free market, and you may say it's not communist/marxist if it pleases you but it's a communist party controlled tyranny. *
If anything, China is much, much more tyrannical and dictatorial than USA. Tyranny and brutal dictatorships is only what communism/Marxism have produced in large amounts. The communism/Marxism have produced very little good and very large evil wherever they got foothold.
* A Chinese citizen cannot even move within China without getting permits from the communist politburo approved officials, forget about him/her being able to launch a business freely.
Edit: added * details
[0]: Trump: What I’m saying is that I will tell you at the time. I'll keep you in suspense, okay?
Read more: http://www.politico.com/story/2016/10/full-transcript-third-...
I don't know why you would equate opposition of mass surveillance with "anti-democracy propaganda". A major point of opposing mass surveillance is that it's anti-democratic for the government to be able to dig up dirt on anybody with the press of a button.
There’s been a sense that communism as a goal in China has been in zombie mode for a while with the system continuing temporarily until an economic crisis or somesuch brings it to an end.
This reminds me of the various “rationalisation” initiatives undertaken by state communism(s) in their early days which formed such a big part of what we think of as communism. These were (attempts at) nitty-gritty policies aimed at achieving the goals of big talking communist slogans and utopianism.
It seems revivalist.
In some ways, this ideas kind of makes “sense’. Digitisation, social media and other features of modern societies lend better (at least in theory, while wearing our 99 year old state-communism hats) to this sort of societal engineering than their 20th century equivalents.
The end was one of the bloodiest crimes against humanity and one of the most widespread genocides of the previous century.
Oh, like the US.
They're searching for enemies of their state and they will probably find them. Let's not ignore the fact that heroes and patriot's often start out as an "enemy".
Sarah Winchester believed she was haunted by the men, women, and children killed by her husband's invention. What will keep us up at night in the future?
http://www.hrnk.org/uploads/pdfs/HRNK_Songbun_Web.pdf (PDF with more details)
This is one of the best Freudian slips / unintended puns that I have ever read. Damn right he's not taking the high-speech train anywhere in China by standing on a soapbox and complaining about Big Brother and disrupting the harmony of society.
If we hypothetically assume that the system is hacker-proof enough and accurate enough for all practical purposes, is there still an underlying moral reason not to implement this system? I instinctively feel that it is wrong, but I just don't know what it is.
I think we all unconsciously rate people in our social interactions. Maybe to explicitly quantitate that rating solidifies it and makes social mobility harder? Our fuziness in our unconscious rating of people gives leeway for others chances to redeem themselves or not? Whereas in a system like this, people are more likely to be stuck where they are?
Facebook/Google are already working with credit agencies and law enforcement.
We make China out to be the big bad guy, but all they do is follow our example.