Deciphering China’s AI Dream
fhi.ox.ac.uk
fhi.ox.ac.uk
Strong rhymes from the 80's and 90's when Japan was the boogieman. Replace Japan with China to update into the present moment.
1) Fear of Japanese AI was real in the 80's. The fifth generation computers (massive parallelism with prolog programs)[1].
2) Fear of Japanese buying the rest of the world [2] and unfair trade tactics.
3) Trump attacking Japan for trade and considering a presidential run with trade war and tariffs [3].
4) The war with Japan is inevitable [4].
5) Under all this Japan had 'demographic time bomb' and real estate boom reaching the maximum.
China will undoubtedly face many of the same problems as Japan did, but the scales are different (both time and magnitude). China has still at least two decades to go. The country is still undergoing urbanization and has to fit over 100 million people into cities. While Chinese growth can't be as high as in last decades, their growth will still outpace the West for a long time.
It's very unlikely that China just fizzles and stays paralyzed for more than a decade like Japan did. Either Chinese solve their problems or the problems will destabilize China internally.
---
[1]: https://en.wikipedia.org/wiki/Fifth_generation_computer
[2]: http://www.businessinsider.com/japans-eighties-america-buyin...
[3]: https://www.washingtonpost.com/business/over-four-decades-tr...
[4]: The Coming War With Japan by George Friedman , Meredith Lebard https://www.amazon.com/Coming-War-Japan-George-Friedman/dp/0...
We are currently living in a era where the results from Hinton & Canadian Mafia are still begin adopted and refined. It's just alternating layers of affine transformations and nonlinearity with lots of tricks and improved routing.
There needs to be fundamental theoretical breakthrough in every decade or so, and not just refinements and new applications. Hinton made his "What is wrong with convolutional neural nets" speech years ago "https://www.youtube.com/watch?v=rTawFwUvnLE I believe that Hinton's capsule networks type attempts are what we need more. They may not deliver immediately impressive results because fundamental research involves lots of setbacks.
1) hype and expectations <= reality constantly
and
2) R&D time horizon <= investor time horizon constantly
Has that ever been the case? Technology advances but there are always periods of over-investment and lost fortunes. Dot-com bubble didn't burst because internet was a fad, it bursted because expectations and reality did not sync. It took over a decade for Amazon stock to return it's previous level.
There has already been two AI winters one in early 70's and another in late 80's early 90's.
and a computation is just 0s and 1s, with lots of if/then statements.
the point is: anything complex can be dismissed as "just x,y,z" if you dont appreciate the massive body of work behind it.
i made that point because OP observed that "ml is just affine transformations" or something to that effect. yes, that's one way to frame it - if youre okay overlooking roughly 30 years of research.
I'd strongly recommend checking out Peter Norvig's talk on "The Unreasonable Effectiveness of Big Data"
You're comparing the hysteria around Japan to the hysteria around China. If you compare the facts about Japan to the facts about China, you get a completely different picture.
Facts:
1. China is implementing a massive military build.
2. China has started to expand its territory.
3. China has repeatedly hacked the most sensitive US gov departments and industries.
4. China recently had an all powerful dictator rise to power.
5. China has covertly supported North Korean's ICBM program in a proxy war against the US.
6. China has allied itself with Russia against the US.
China has not supported North Korean ICBM program. North Korea apparently has blueprints of early Chinese nuclear design, but they received it trough Pakistan. North Korea has received more help from freelancing Russian rocket scientists than it received from the China.
China is not allied with Russia. At best they are exploiting the conflict between Russia and US to get military technology transfer and cheap energy from Russia. Russians don't trust Chinese and the feeling is mutual.
Chinese also immigrate into other countries in vastly higher numbers than Japanese.
The japan, china comparison is close to false equivalence
The latter attribute, creativity, is something I have always thought was necessary to foster with freedom in thought and expression, two things I see incompatible with the authoritarian system currently in place in China. Maybe I'm naive and romantic about the Canadian/Western style of education and China will inevitablly dominate AI through force of will. However, my personal experience working with it, and experimenting with improvement in my own small corner of applied science tells me otherwise.
Would be very curious to hear others thoughts on this, perhaps from people further along than I (still a lowly undergrad).
Edit: I don't want my comment on math to imply that there is no creativity within mathematics. I've gone through too many heavy math courses to believe that. I'm instead trying to say that there is a place, even beyond mathematics, for creativity and intuition, where you don't need the mathematical understanding to build something novel.
The bigger problem is just talent based: china isn’t taking much advantage of non chinese talent, while America is. The American talent pool for top tier talent is simply much larger than china’s despite having a much smaller population.
Follow up: Do those people then get encouraged to continue to foster their creativity within the system? Does the government, upon seeing that someone "withstood" their education change their approach and allow them to continue to explore?
I would say Japan and Korea are much worse than China, with very restrictive corporate cultures that are actually more Confucian than China.
Intellectual property management is archaic. And your other claims are borderline racist.
Your priors are wrong. I work in the US, public and private grants are shrinking year over year, and anti-intellectualism is rampant. Meanwhile my Chinese colleagues, at the same place in their career, are practically local celebrities and receive unprecedented (in the US) resources to do decade longitudinal research and increase headcount. In reality, China is on the bleeding edge in AI because their government and private industry has the social cohesion to focus on scientific advancement. Besides the vast intellectual capital at their disposal, they can cheaply afford to iterate quickly as the means of manufacturing and production are local.
So, because I don't have the right experience I shouldn't voice my opinion, in a way that suggests I am willing to be proven wrong? I never said I was correct, I said this is what I think and wanted to know what others thought. I never claimed to be qualified, I actually implied I wasn't. It seems you have more experience in this area than I, so I learned something from your comments. Again my issue with your comment isn't that you're telling me I'm wrong, its that you are telling me I am racist
Where China may fall short is actually when people become overly "creative": authoritarianism conditions people to individually work around rules they don't like instead of working together to change the rules. When rules are ignored you have rampant corruption and heavy pollution despite rules on the book.
There are certainly elements of jingoism at work here, too.
I'm trying to figure out what your comment means.
Deriding culture on the basis of tenuous claims is a deceptive way to evade accusations of racism.
There are many things that the Chinese government is fantastic at, for example energy. Their transition to green energy has been fantastic for both the industry and the planet as a whole, and I wish other governments would follow their example. If I critized the green energy policy of the American government would I then be anti-USA?
Generalizing and taking arguments to extremes is something I didn't expect on this forum and am dissapointed to have to respond to.
This is such oft repeated nonsense. Creativity exists in authoritarian systems. Creativity existed during slavery. Creativity existed during the dark ages. It existed during the age of kings. It existed during religious tyranny. It always exists.
> Maybe I'm naive and romantic about the Canadian/Western style of education and China will inevitablly dominate AI through force of will.
What is canadian/western style education? It's so funny how we mock our education system as producing factory drones incapable of thinking but whenever china is mentioned, we pretend like our education system fosters creativity. People constantly regurgitating the trope about "creativity" and "china not having it" just shows how well brainwashed we are. I wonder how the chinese built their own unique civilization if they weren't creative?
> Would be very curious to hear others thoughts on this, perhaps from people further along than I (still a lowly undergrad).
It's a simple matter of wealth. If china can develop enough and get enough money to be able to spend time doing the research, then they will make progress.
It's simply a matter of wealth, people, resources and stability. China has the people obviously. They have stability so far ( I doubt anyone is going to invade them and destabilize them a la opium wars ) and they appear internally stable as far as any can tell. They have a large enough territory. So it's simply a matter of wealth. Can they get wealthy enough to build universities, companies, etc to fund research?
Creativity, research, etc has nothing to do with the political system. The authoritarian soviets were able to send people to space. Nazi germany was able to advance research on everything. So did imperial japan. The authoritarian and tyrannical european colonial powers ravaging most of the world made great contributions to science/knowledge. So did we while we were busy exterminating the natives and enslaving blacks.
The system doesn't matter. People are just as creative in china as they are here. The system doesn't matter. It's a matter of wealth/funding and stability.
In terms of your question on education, I have only experienced Canadian education but I have never thought of it producing "factory drones" , exactly the opposite. Some of the most creative thinking I have seen has been in the classroom, with professors/teachers asking us to point out their flaws, or think of better answers than they presented.
China still has a lot of creative people, but the structural differences of the Chinese education system can’t be ignored, and I cringe whenever politicians say we should have schools more like theirs.
Example: Shanghai (25 million people) decided to do something new. They learned from other countries with good PISA results and adopted it to their system. Less tests and more long term projects. Little more autonomy for theaters and better rewards. Their experiment was a success and they reached the same level in PISA studies as Finland and Singapore.
But I don't see how performance on an exam relates to creativity at all.
The condition for enrolling in public schools: http://www.shanghai.gov.cn/nw2/nw2314/nw2319/nw12344/u26aw55...
The condition for obtaining residency registration: http://zwdt.sh.gov.cn/zwdtSW/zwdtSW/juzhuzheng/lsjzz.jsp
e.g. in my area, say kid's dad has hukou in the local area and kid's mum has hukou from another area in Shanghai, then the kid will get lower priority for applying schools in the area comparing to those with one of the parents with hukou in the area and another one without hukou in Shanghai at all. The logic is that the first couple has the option to send their child to a different area in Shanghai while the second one doesn't have such option available to them.
The results of creativity spreading to the economy can be hampered by the government. But I strongly disagree that creativity and innovation can't flourish under authoritative regime.
Educational system does not create or remove creativity, it's just a common myth. The reason why Chinese students look so mediocre is because they value science and engineering so much. It's seen as a path to good live. The creative people are still there, but there is more mediocrity because there are so many of them.
China: 4.7 million STEM graduates per year = 34 per ten thousand.
US: 0.56 million STEM graduates per year = 17 per ten thousand.
Chinese have adopted market mechanism and Chinese companies compete on global free markets. Government is just coordinating and supporting the industry just like DARPA is doing in the US.
And I maintain that this is just a story we tell to ourselves. Can you find anything to back it up.
Two reasons for this.
1) Different cultures express same things differently and get same results using different methods. Direct challenge is not the only way.
2) Chinese are not especially obedient or submissive. Regional riots and unrest that you don't see in the news happen regularly. The way people solve their problems with government is very different from the West. Chinese priorities are in getting ahead in their life, not in political activism. If Chinese government fails to deliver economic growth they will have to deal with more rebellious people.
Using Hoftedes research to compare different cultures: China has similar levels of individualism and their long term orientation is similar to Germany, South Korea and Taiwan, but their cultural uncertainty avoidance very low. It's lower than US, Germany South Korea or Japan.
ps. Cultural comparisons in social sciences contain lots of BS but Hoftede's research is relatively solid. His research in IBM is classic. It has limitations but it is useful.
Looking from the west China looks like solid block and Han Chinese who make up 92% of the Chinese population look even more so. Chinese government want's to propagate this view of unity. Maybe Beijing and Canton are culturally more like Germany and Britain or like South Korea and Japan.
What is an advantage however is the US culture which promotes a certain type of courage. In degenerate cases it combines with bad education to become a naive form of arrogance and self indulgance. In the most actualized forms it combines with strong work ethic and refusal to look away from reality to create very big change. (ironically this freedom culture combines best with immigrant mentality)
The Chinese are very creative, except this creativity is often expressed in trivial ways because the culture creates a sense of "fineness" with things. The US culturally has what the Chinese refer to as "the heroe's dream" where everyone imagines themselves the hero. I suspect Chinas large population as well as it's more authoritarian government plays a role. Historically pre-communist era even the "important" things were left to the imperial elite.
This is hard everywhere. I'm not sure who said, "knowledge advances one funeral at a time," but I'm sure that it wasn't Confucius.
In a political culture that largely will prevent outsiders from participating in business or non-tech academia inside the country (who will want to live/teach inside China, micromanaged by Big Brother's social score — other than the Chinese?), I think China's heavyhanded political leadership of the future will form an insurmountable obstacle against the country's ability to compete in any/all of tomorrow's critical revolutionary exponential multiplier business spaces — the ones that silicon valley startups excel at, that require dynamic multicultural madhouse environs, and that engender future Googles or Apples. Without the freedom and chaos needed for this space to thrive, China will forever relegate its rising business geniuses to pursue mere optimizations of The Next Big Thing, that will be invented where new ideas are free to run amok.
In the not so distant future, China's cut-rate factories distributors and e-tailers will be automated, like everyone else's, and her political stranglehold on the genesis of new ideas will lead to her uncompetitive downfall. That is, if a billion unemployed don't do it first.
Thanks to capitalism - they can just buy makers of competitive products. Have you looked at the 2018 Volvo lineup? I think they are fantastic cars.
The US is a couple of education reforms away from being a hero generating lottery to a hero generating machine. Also the fact that China is pretty much closed to foreign outside heros ever getting a chance.
It's always dangerous to believe something based on wishing it were true.
Yes, China is an autocracy, and yes autocracies might hamper scientific and technological development (though there are many counterexamples). It would certainly be very convenient for the West if China's growth faltered because their political system can't do AI.
But China has an impressive record of economic growth over the last 30 years. In a decade it will probably overtake the USA's GDP at current exchange rates to become the world #1 economy; with 4 time's the USA's population, China doesn't need as high GDP/head to have a total GDP much higher than USA. A rich China means they will have plenty of money for education and AI. Chinese people already score higher in IQ tests than westerners. It is quite likely that they will start selecting embryos for intelligence soon, and then there will be a bigger IQ gap -- with more people who're more intelligent, China will have a massive advantage in AI research (as well as everything else).
It strikes me as entirely plausible that China will dominate the world in the 21st century, in AI research as well as economically and militarily.
It’s not that there wasn’t creativity, but the culture very strongly did not allow for anyone to perform duties or tasks even moderately outside of exactly what they were told to. I think this definitely has and will hold them back.
From p.18 of the report:
"What is new in the hardware driver is that Chinese tech giants and unicorn startups are competitive with some of the world’s leading companies in designing AI chips. For instance, Chinese company Cambricon, a statebacked startup valued at $1 billion, has developed chips that are six times faster than the standard GPUs for deep learning applications and use a fraction of the power consumption.65 Moreover, equipped with a new “neural processing unit,” Huawei has arguably overtaken Apple in mobile AI chips.66"
So far Nvidia has build microarchitectures that are generic. They just put tensor cores, F64, F32, F16 units and 8-bit inference support into the same GPU. They are used in HPC, graphics, DL inference and training. That's four different domains.
I suspect that in the future Nvidia builds 2-3 different microarchitectures.
One for HPC + graphics.
One for DL training.
One for DL inference.
https://web.archive.org/web/20180318140130/https://www.fhi.o...
EDIT: Hrm, guess it only cached the abstract.
https://www.fhi.ox.ac.uk/wp-content/uploads/Deciphering_Chin...
Mirror: https://archive.fo/QhRnC
As one AI researcher said, the temptation to try it just too sweet. And China's motto of already "once untrustworthy, always restricted" for the social credit score system isn't very encouraging.
If it can control people efficiently but has less negative externalizes than traditional authoritarianism, it will eventually spread outside China.
Resilient, stable and productive authoritarian system would turn the tables between liberal democracies and the rest of the world. It would be a pivotal moment in human history.
This is a myth created by inaccurate reporting. For details on what the policy is exactly see my earlier comment:
And I'm confident the goal will be reached. Incidentally, much of the software that today is classified as "just software" will be called "AI".
Not that it's different from elsewhere, of course (I saw screenscraping being called "AI"), it's just there are government incentives to do so.
The financial system is different. In the West, there's no central authority requesting what needs to be done, but small and big companies are allowed to go crazy with user data, as long as they provide access to government agencies too. In China, the whole thing is managed by governmental entities, but it functions strikingly similar. The Chinese also lack a lot of the tact and PR management that western companies are so good at - but the differences aren't that big in the surveilance. One other key difference is that the Chinese aren't afraid to prosecute people based on that data, while in the West we still have some due process, but the future plans aren't looking good.