Beijing to Judge Every Resident Based on Behavior by End of 2020
bloomberg.com
bloomberg.com
frustrated lately that my rating as a
passenger in Uber is at 4.78 and falling
I mean, of all things, I don't even like the HN karma system. Arbitrary frowny faces for The Wrong Kind of Joke only now, scale that up first, just to taxi drivers, and then apply it to Society At Large, All Day Every Day.
Is it possible to fix this? Am I going to have to change my name, email, and phone number before I can use Uber again? And how is this AT ALL better than the old taxi system, where the taxi company owned the car and was able to insure itself against this kind of damage caused by passengers? Not to mention that if Uber is successful, it's likely to become the only game in town - whereas taxi services are quite diverse by comparison.
Also, why do you assume that I have to be a "bad person" to get a bad rating? What if I'm a minority living in an area where my background is looked upon with suspicion or hatred?
It is crazy the latter could land you on a blacklist. A rating system is not necessary. A reporting system for the rotten apples would suffice, and the rest of the customers would be unrated.
The guy brought to hospital may end up in coma. Not able to clean up.
With reporting you cannot have a pissed-off driver land a pregnant women on a blacklist because she delivered her child in the car, and there is now blood on the seats.
There is some occupational hazard for taxi drivers and others in public transport. The provider company or insurance should cover cleanup costs.
Every consumer interaction now rates you as a customer. Amazon will drop you if you return too many things. Uber/Lyft have ratings systems. etc.
Until we start making the holding of personal information the equivalent of holding toxic sludge, nothing will change.
You know what you did wrongly to lose points, so that you can avoid it next time.
You also know what are good things that you can do to improve your score.
https://www.hrw.org/news/2017/12/12/chinas-chilling-social-c...
Then the rules system is very complex, and probably will be ever changing, being fine-tuned. There are rules in preparation (or already implemented) where your network of friends and relatives affect your credit score, etc. Very dystopic social engineering.
It will be harder and harder to check whether all rules are applied equally across all people, I imagine.
> There are rules in preparation (or already implemented) where your network of friends and relatives affect your credit score, etc.
Are these your opinions or facts? I'm talking about facts here.
https://duckduckgo.com/?q=china+social+credit+blacklist&t=fp...
There are also many mentions of your friends network affecting your score, like in this Wired article from 2017:
> Zhima Credit’s algorithm considers not only whether you repay your bills but also what you buy, what degrees you hold, and the scores of your friends.
I think you misunderstood what this means. It means when taking loans, the algorithm takes into account your score as well as your friends. This assertion in itself I must say I didn't find from any other sources.
This, even if proven to true, is completely different from what you are suggesting (i.e. your friend's score affecting your score).
Also, this Zhima Credit is just a private owned, part of the system which focuses on financial well being of individuals, it's not linked to other component of the system. Other social score components might not have this network effect.
> But that is not the half of it. Scores will also be awarded to citizens according to who they associate with. If your social media connections consist exclusively of senior government officials and respectable party ideologues, the chances are the presiding algorithm will allot you the equivalent of a triple-A rating. But if you happen to have exchanged messages with someone who is friends with a human rights lawyer or the imam at the local mosque, you could rapidly find your status cut to junk.
https://www.scmp.com/week-asia/opinion/article/2131737/china...
I was looking for another article that explains it in more detail still, but cannot find it. But have seen this described in many places. I haven't been to China, and can't tell how far they are in their implementation. Just going from news sources.
2. The author seems to reference the official document from the government. I did an extensive review of official document on social credit system (link in my profile), and didn't find anything describing such thing.
Maybe it is just a feature of the Zhima Credit (a private financial credit system), which happens to be included social credit system.
To me, as a lay man reading about all that is planned already, it does not seem all that far-fetched to see this being implemented soon. But as you rightfully say: it's the facts that count.
Except if you don't agree with the definition of "wrong" and "good".
THAT you can't correct.
I need to bookmark this.
Because in China, it would be the government giving you the rating, not other people.
It almost goes without saying that such a system is ripe for abuse and puts in to the hands of the powerful a new tool for control over the powerless.
The average citizen will now have to jump even higher when their masters tell them to jump, while those at the top of the political hierarchy are very likely to be exempt.
The Blanks were people who have no record in the system.
Now that was a prescient TV show if I ever saw one.
[1] - http://www.maxheadroom.com/index.php?title=Episode_ABC.1.6:_...
[2] - https://en.wikipedia.org/wiki/Max_Headroom:_20_Minutes_into_...
Naively thinking this time it will be different is not understanding human nature.
And don't forget: If you are a model citizen, then it is all fun and games. You'll get benefits instead of punishment. The apps you use are highly gamified and use all the persuasive / behavioral tricks of the book. Many Chinese truly love the whole system. They think people with low scores + punishments really deserve that.
This is the same government that has killed 40-80 million of its own citizens since taking power in 1949. With impunity, if that was unclear.
The repression has lowered considerably during the economic boom of the last 3 decades, but it's naive to compare this new system to some liberal utopia. That is not what it's replacing.
The fact that Mao tried to have Deng assassinated goes to show that it became a different government when Deng ascended to power.
The fact that Xi was sent to a reeducation camp in the 70s and now he is the president of China shows this is a different government.
You can think of it as the same government, just ruled by a new set of people. It's not like there was a violent overthrow. The Party is still very much in control.
But these categorizations don't mean all that much.
I hope I'm not being too cynical by saying: the things we fear most about this system are not abuse, they're the system working as intended.
Xi is a dictator and he is riding to and fro on a tiger from which he dare not dismount. Mass surveillance and a strong police state are the tools he uses to stay in the saddle.
You raise a good point, but I’d still be wary of both possibilities. Or some hybrid mix where it’s the worst of both worlds simultaneously.
The Chinese system seems a whole lot more dystopian since it’s state run, and since they are trying to link unrelated things — i.e. penalize you for certain “anti-social” behavior that goes far beyond having poor credit.
This is already how it works, more or less, but when such credit rating becomes (a weighted) part of a social credit system, then it hardwires wealth inequality. A rich person can do a lot more 'wrongs' before low social credit becomes a problem
Anticipating pushback — yes, our healthcare system is atrocious, but the "all bankruptcy is medical bankruptcy" thing is just a meme: https://www.washingtonpost.com/blogs/post-partisan/wp/2018/0...
Also, I know that people are bad at thinking about debt, so I don't morally fault them for taking on more than they can handle. I've spent beyond my means before! That said, society needs some level of personal accountability in order to function.
While I hate this system for privacy reasons, there is always going to be some sort rating system for businesses to avoid high risk customers.
no lease, no credit card, no nothing. America gave it to the corporations. So what difference does it make if an all mighty corp does it or the gov't?
This system if it is unstable and has side effects will probably screw up two generations at least.
The main issue I see is that people with low scores can “infect” other people’s scores. Considering this like a viral phenomenon the score will be impacted starting with family, friends, colleagues, strangers. I can’t see a solution except going the old route of “killing the nine family relations”.
I can see some kind of ghetto of low social score people doing barter and what not.
There are some contradictions like donating blood giving good score but what if for someone with a low score. There goes empathy if punished.
Or some exploits like colluding and creating cartels of increasing social score artificially.
This will also make you reachable even if you're not living in China. Say something China doesn't like? Your family and friends score suffers in China. Do it for too long and they disown you for their own self preservation.
But who knows, maybe this will work out and be a net positive for society at the cost of the few people that will fall through the cracks and be evaluated unfairly. It seems like this is going to happen one way or another, so here's hoping for the best.
Let's start with news aggregators to talk about ideas winning out on a massive scale. One thing news aggregators do is act as a sort of activation function for ideas. For a given news aggregator an unpopular idea within the group will get tamped down while a popular idea within the group will get magnified. For example, an article about how Rust is awesome will get more points on Hacker News than an article about how Rust isn't any better than C. So when we use news aggregators we learn how to be heard and what to avoid. It's almost so bad that you can see an inaccurate version of history being written before your eyes sometimes. For example, during the last presidential election Reddit was awash with extremist left wing propaganda because those ideas were rewarded and magnified on that site. Meanwhile any centrist or right idea was immediately tamped down. The aggregator has spoken! Again, we see what to avoid and what to do to get a reward. So we trade our autonomy for internet points. I guess that is why they call it the hive mind.
To me it seems like our ideas are becoming more unified even as our population expands. I think this is because ideas spread easily on social media. But at the same time it's just easier to do things at scale. It's easier to build cookie cutter houses than homes for individuals and honestly it's easier to live in a cookie cutter house in a cookie cutter life and give up some of that autonomy so that you can focus on what matters to you. e.g. so you can specialize or just provide the best possible life for your family. But even in specialties there is a strong demand to conform to popular consensus. That's why, to me, it seems like we are outsourcing many parts of ourselves. We give up a little autonomy here so that we can focus our energy elsewhere but when we're expected to give up our autonomy there too we again become nothing more than extremely hard working, unappreciated, and unfulfilled neurons. *At this point it might sound like I am talking about myself or that I hate my job, lol. Just to clarify, I actually do enjoy my job but I know what it's like to work in a web development dungeon.
The last little bit about Orwell I don't think makes a whole lot of sense now that I look at it again. He surely could have imagined a dystopian future where a power hungry government turns its citizens into slaves by controlling their access to resources based on invasive monitoring of their "behavior". It just seems a little deeper than 1984 to me because of the psychological impact of living a prescribed life and carrying around a fake smile every second of the day.
I’m currently prepping for an interview at one of the big tech companies. It’s hard to push aside the thought that I’m a nobody who at best will spend years nipping at the toes of the hyper privileged individuals who are no more capable than me (in theory), yet who, because of their birth rights (parents, wealth, connections, etc), are in a position of power where their subjective desires will dictate what I do and say with my life (at least 1/3 of it, including the 1/3 where I’m unconscious). It’s even harder to think that as time goes on, this paradigm will only grow further. Just thinking about the updates I’ll make to my resume to pass the test, and how I’ll have to squeeze in the right buzz words all while displaying a cheery demeanor as though this is all a favor to me - it saddens me. And then I consider how in 5, 20, 50 years this whole process will be that much more nuanced and invasive...
It’s not the future I saw in youth. It’s the opposite. I thought the world was one where work produced more for everyone. More wealth, more comfort, more freedom. But it seems inevitable that in a finite space with an ever growing population, that’s not going to be the case. It’s a constant struggle for power. It’s not always the same individuals and groups driving it, but the result is the same every time.
Thank you for the write up. It resonated deeply with me.
I’m currently prepping for an interview at one of the big tech companies. It’s hard to push aside the thought that I’m a nobody who at best will spend years nipping at the toes of the hyper privileged individuals who are no more capable than me (in theory), yet who, because of their birth rights (parents, wealth, connections, etc), are in a position of power where their subjective desires will dictate what I do and say with my life (at least 1/3 of it, including the 1/3 where I’m unconscious). It’s even harder to think that as time goes on, this paradigm will only grow further. Just thinking about the updates I’ll make to my resume to pass the test, and how I’ll have to squeeze in the right buzz words all while displaying a cheery demeanor as though this is all a favor to me - it saddens me. And then I consider how in 5, 20, 50 years this whole process will be that much more nuanced and invasive...
This describes me exactly. I've been self-employed for over a decade, I do very well at it, I have a growing "side hustle" helping other freelancers and consultants build stronger businesses, and it's close to taking over as my dominant source of income.
But every six to twelve months I just get really, really tired and I think about getting a job at one of the big tech companies. I think I could probably make as much or more over the next 10-20 years as I will working for myself, and it doesn't seem like the worst gig. It seems emotionally much easier than what I'm doing now.
But one of the biggest things that holds me back is the feeling you describe of just being a nobody cog in the wheel. Can I do that? It sounds crushing, but I can't decide if that's my ego, me being unrealistic, me placing too much value and identity on career and work and money, etc.
When I was in my 20s, I had an opportunity to go work directly for a founder of one of these companies. Everyone knows his name. He was looking for a family tech manager to basically manage all the technology for his houses around the world, yachts, private jets, etc. I deliberated for days about even interviewing for this gig. I did, but didn't get the job in the end. Maybe for the best. But the reason I didn't know if I wanted it was that I felt pretty sure that the billionaire across the table never would have taken that job in his 20s. So why was I?
And one of the things I learned that day after spending an hour with this guy was this: he's nothing special. Yeah, he's smart, and he's worked hard, but I didn't see anything there that seemed foreign to me. And that's why I've been self-employed for the last decade, and why I suspect I'll never have a job again. Because it just seems unbearable to accept defeat and get a job working for someone who got lucky, yes, but also didn't just accept defeat and get a job when it was on the table for them.
Enough rambling, back to work!
Hope things are working out for you.
I appreciate the writeup, I find it really interesting, but I can't decide whether I agree. I'll take a stab at explaining my thought process, but I haven't thought this through very much, so bear with me.
On the one hand, I definitely see the phenomenon you're talking about. The access to info we have, and the speed we have to access it, means that more and more of our lives seems to be driven by feedback loops. We also make it worse by naturally gravitating towards loops (like communities) that reinforce our preconceived notions. That's always been a concern but we have so many more options now, and we can find a slice of the world where there's basically no disagreement at all, and then ruthlessly expel anyone who questions our dogma. I see this a lot in certain subreddits: insta-ban if you reasonably question the gospel they espouse. I don't know, maybe there is room in the world for conversation spaces that aren't constantly subjected to the disagreement of outsiders, but it's also a little alarming to see it spreading so fast.
On the flip side though, here are two things that give me pause about your theory.
First, no matter how unfashionable they might be, the laws of physics still operate. Everyone might be flocking to some new-fangled framework, but if it's clearly inferior to another, then there's an opportunity there to go against the grain and win. It doesn't always happen, but it's not that rare. Ditto for your political example: Reddit was (and honestly still very much is) an embarrassing echo chamber of leftist ideas, with very little tolerance beyond a few subreddits (which usually have their own intolerance, just on the other side). But Trump still won! The echo chamber has influence, but the laws of physics ultimately don't care that much.
Second, technology does allow for more individualism. Maybe most people are becoming more "standardized" within the big groups, but others are able to be more honestly who they are, aren't they? So maybe the world is splitting into two parts. The much bigger part is becoming more homogenous and clustered, and then a few people are becoming more individualistic and creative and able to chart their own path.
I don't know though, as I'm writing this, I'm wondering if those people aren't actually more individualist, they're just in a smaller group that behaves the same as the larger one. But is that driven by technology? Or have we always been like this, and technology is just making us better at being terrible?
Anyway, if you're ever in NYC, I'd enjoy chatting about this over coffee or beer :)
This seems likely to cause those who don't want or can't fit into the rated society China is building to emigrate.
The more creative, the misfits, those square pegs who don't fit in round holes... Send them our way.
Meanwhile, the algorithms used to make these decisions are often an unaccountable, un-analyzable black box. It's going to be very possible to fall through into an "undesirable" category without any obvious misdeeds or obvious ways to fix the situation. I expect we're going to see a category of consultants emerge that will help you "fix" undesirable algorithm ratings - just like we have for credit ratings.
[1] https://techcrunch.com/2016/08/15/naborly-lets-landlords-scr...
[2] https://venturebeat.com/2018/10/04/babysitter-screening-app-...
[3] https://medium.com/privacy-international/social-media-intell...
This is a HUGE difference because you are allowed to exit a given system / rating scheme or not use the services that require it. That may be difficult, but it's not literally prevented by the government. You and your fellow defectors can build alternatives. Build your own institutions.
People have literally no idea how good they have it. If they think everything is hell, why would they fight to protect it?
No, it makes us comparable. Not comparable like "we're the same". It makes us comparable like "we have something similar and let me show you some of the differences and correlations so that we might learn something".
> People have literally no idea how good they have it.
I think you're observing us focused on the bad and it makes you think we've lost sight of what's good.
> If they think everything is hell, why would they fight to protect it?
What right do we have to something which we do not wish to improve? How can we fight to improve if we do not discuss how it's bad and what could be done to resolve the problems? How can you resolve problems if you don't recognize them in the first place?
That doesn’t mean that they’re all “comparable” in any reasonable sense of the word.
Is a small tribe of Native Americans in the 1400s "comparable" to the European Union in 2018?
Only in the sense that you can compare them, but then, you can compare anything, so literally anything is comparable in that sense, which robs the word of all meaning.
I think there's quite a bit more in common between a 1400's tribe and the EU than say, the star Polaris and trends of Swedish Cubism in the years 1950-1980.
The point here has always been that throwing up your hands and saying the original comparison is unreasonable is not a reasoned argument, it's flipping the table and walking off. The systems of social control in the United States and China are far more comparable than "anything to anything".
Following novelty can lead to many disasters.
I imagine that suicide and such self destructive behavior will skyrocket.
In the US, if a citizen is suspected of getting out of line, all social media data, phone metadata, all bank transactions, etc., for that individual and all of his/her friends or acquaintances, which for most people is tens of thousands of people, is available for review by law enforcement, with no doubt that a warrant will be obtained.
FWIW, in the US, anyone within 100 miles of a border (which is the majority of people) has no 4th amendment right against arbitrary search.
There is most definitely a social credit system in the US to the extent that one is necessary, but due to the success of other social control mechanisms it is not necessary for it to be as overt as China's.
This sort of anti-China propaganda has been building over time and this story has been tested and revised a few times, but per the article below, is not actually true:
https://www.brookings.edu/blog/order-from-chaos/2018/11/19/c...
You’re wrong about there being no doubt that a warrant can be obtained to examine the records of tens of thousands of people because one person “got out of line”, except in the most extreme cases. Well, depending on how you define “getting out of line”, maybe extreme national security but that’s relatively rare. I’d like to see a source showing that such warrants are widespread.
And that’s not how the 100 miles from the border / 4th amendment thing works either. Not that it’s not a complete clusterfuck, but it’s not as simple as you describe: https://www.aclu.org/other/constitution-100-mile-border-zone
Edit: to clarify for the dead comment in response to mine: America does all kinds of terrible shit. We always have. Internment camps are a stain on this country. And the stuff we do in the name of national security spits on the spirit of the constitution. But that doesn’t mean it’s on the same level as all atrocities by other governments, or that we’re robbed of our moral authority to condemn human rights violations because we’re imperfect. And the lazy whataboutism that’s dragged out every time there’s a story about a truly horrific authoritarian regime trampling on its citizens is galling. It’s ignorant and useless.
I think you've hit the nail on the head by describing it as a form of moral authority. American exceptionalism is a religion, and the idea that America's actions are morally justifiable no matter what their humanitarian cost is one of the fundamental tenets of the religion.
To be clear, American exceptionalism is among the most destructive and horrible religious regimes ever to exist, and has caused untold suffering all over the modern world. Keep in mind we live in 2018 when most disease and scarcity that once prevented human populations from flourishing have been eradicated.
Yet America is engaged in countless wars abroad and coerces its allies to collaborate in the enactment of inhumane sanctions aimed at all sorts of populations of innocents all around the world. Their crime? Not giving up their lives to overthrow whatever dictator has taken power.
Meanwhile we sit back and watch it all on CNN or Fox and praise the bold, morally principled actions of our political leaders whose moralizing led to the sanctions, bombs, drones, etc.
It is absolutely unconscionable for Americans or American leaders to attempt to use some sort of moral superiority as a justification for committing these kinds of acts. Why is this story being promulgated about China in spite of being oversold? Because some American leaders want to threaten China militarily and want to disturb the peace that has been achieved over the past decades.
One reason American Exceptionalism may not seem to you like a religion is because it is so unbelievably widespread. Politicians are expected to pepper their remarks with platitudes about the moral righteousness of our wars and our policies. It's all complete nonsense.
We act in the service of powerful interest groups, aligning with whatever dictators suit us at the moment, and never will our political leaders discuss any sort of definition of national interest. They prefer to talk in religion-esque language with platitudes about how brave our soldiers are and how virtuous our motives.
But in fact, a simple clear articulation of what they claim US national interest to be would be more than enough. If it's in the US national interest to somehow engage in conflict with China, then claim it outright, don't spend years trying to paint China as totalitarian or ultra authoritarian. It's not true nor is it remotely relevant.
The real cost of American exceptionalist dogma is that so many of us never think deeply about why we are supporting various wars or sanctions or dictators. We just assume that of course the US would never do anything untoward while our leaders exploit that naivete to the fullest.
So please spare us the platitudes about WW2, the so-called greatest generation and all that nonsense. War is ugly and we most certainly did not enter WW2 for humanitarian reasons.
If the consensus mechanism used in a democracy is a simple majority, that leaves the opportunity for 49% of the population to be entirely disenfranchised by 51%. Let's assume this is fair and reasonable for now.
Moral authority is something that has traditionally been claimed by members of the clergy, and notably by many governments in which the political leaders are also the religious leaders.
We've seen the many downsides of theocracy around the world, so I find it concerning that so many Americans are willing to accept hollow moralizing from elected leaders.
If a president said, "I woke this morning and I knew that God had spoken to me. He said that the US must invade country X. And so it shall be done", most people would all be appropriately freaked out.
But when instead the political leader says "Country X is led by a mentally ill tyrant, women of country X are oppressed, and the infrastructure in country X under the current regime is failing, resulting in high levels of disease and infant mortality", everyone applauds the upright moral character of the American politician.
Every day the newspapers contain a number of articles pointing to infrastructure problems in other nations... whether it's drug violence in south or central America, corruption in India, the smell of garbage or sewage in China, etc. Not to mention the many articles taking aim at social ills blamed on the ruling party in any number of nations. This writing is read by Americans as we read celebrity gossip, as entertainment. It reinforces the narrative of American exceptionalism.
But one might ask why it is necessary. It is done to help make Americans view foreign people as beneath them, unable to prevent drug violence, corruption or the smell of garbage. We are meant to read these articles and think "wow I am sure glad I live in the US".
But in reality, the way of life for middle class people in many countries in the world is superior to that experienced in the US. Not in the way that is reflected by per-capita GDP, but if you go there and see how people live it is obvious.
If Americans realized that we would tolerate a lot less from policymakers. We might even insist on accountability for grand ideas that promise to be money pits. If we saw the people of other nations as getting a lot of things right, maybe even getting some things more right than we do, we might be reluctant to allow our leaders to fly drones over their countries or scapegoat their exporters.
The entire American security state and massive military projection of power could not exist without the opiatic effect of the state religion of American Exceptionalism.
So while the program going on in China may be of some minor level of concern, we need to see through the PR campaign that is going on and realize why we are being told about this one narrow aspect of Chinese policy.
[1] https://en.wikipedia.org/wiki/Project_Cybersyn
To the unknown person who criticized me ("people like you are what's wrong with the world") but later deleted his post:
To see this from a mere political angle, is to miss the point entirely. You are deluding yourself if you think that similar cybernetic feedback loops are not currently in play in western democratic states. The difference is that in the west, these loops are driven by non-state actors and focus on cultivating consumerism (look at the Black Friday hordes), apathy and lack of critical thinking. By taking an active approach, the Chinese state could try to pre-empt these problems on one hand and aim at creating a different kind of citizen on the other.