Power and Paranoia in Silicon Valley
pdf.yt
pdf.yt
The people wandering around claiming that AI will take over the world soon have no idea what they are talking about. What is actually happening is the second wave of hype in AI. The first wave, in the 80s, ended up as the "AI Winter" when people realized that AI was not going to take over the world. This second "AI Winter" is going to end up causing another 30 year research desert. People have such short memories.
Here is a way to think about AI. The best known systems are actually people. Imagine you had invented a person for the first time. Would it take over the world? Probably. But it would likely take tens of thousands of years. And this new "person" would be prone to faults and failures, such as when it can't remember where it put the keys, because some of these faults and failures are inherent in the system.
I don't fear AI. I fear people who have no idea what they are talking about who "pump up expectations". Get all the funding you can now because a new AI Winter is coming.
The question you should be asking is why do those people believe in the singularity? The answer -- like so many answers in life -- comes from the realm of psychology and sociology rather than hard science.
Ok, as long as we're bothering to make this distinction, I don't think that's correct. Most singulatarians I've hung out with place the locus-of-control in their story internally: they believe that, as a group, they will ensure that The Singularity comes to pass, in all its Charles-Strossian-weirdo-glory. As detailed in the article, they have actual groups that are trying to make this happen. They can be incompetent at it, but you can't say they're sitting around waiting for someone else to save them.
They're more like Communists than Christians.
>the difference between this religion and other, older ones is that older religions focused on morality and human nature, while this one is not even about humans at all.
If you followed the article, the whole movement we're talking about has a loud, public component called Effective Altruism who follow Peter Singer's (noticeably Christlike, for all I'm not at all Christian) idea of giving away large portions of their personal income to the highest-impact disease-fighting and antipoverty charities operating in the Third World. Extreme, impersonal charity might be inhuman, in some degree, but I'd be hard-pressed to call it either immoral or disconnected from moral concerns.
>The part about the guy weeping because previous generations didn't have Kahneman was clearly written to reflect Christian sorrow for the generations prior to the coming of Christ
A guy wept because previous generations didn't have Kahneman and Tversky's foundational findings in behavioral economics? Excuse the following several minutes of nonstop giggling.
Sorry, now, you're telling me it's patterned after some Christian thing? I can't really tell yes-or-no here, since I've got no background in Christianity.
>but the latter group is sad about those who didn't hear Jesus's message about kindness and love, while the former was crying about those who couldn't tweak their minds to be more robot-like (not that he can, either, but at least he imagines he can).
Well, this provides a good opportunity to sum up, and to mention a couple of things.
Firstly, I too find it funny that people want to tweak themselves to better operate as fully, calculatively rational utility maximizers. There's a whole literature of bounded rationality showing that throwing out one's information and computation bounds and just trying to approximate unbounded rationality in finite space is a very bad idea ;-)!
But secondly, just saying, if people once got sad about others not hearing Jesus's message of kindness and love, well, what of these people and Peter Singer's message of realizing kindness and love in the here-and-now?
Or, to get one level crazier, what of the people going to Secular Solstice and weeping that mankind has not yet defeated death, and that we will all lose our loved ones, suffer ourselves, and then die ourselves? I rather imagine that the single fundamental fact of the human condition as we know it is an entirely appropriate target for religious yearnings!
The Buddha said, "Existence is suffering." The extropians said, "Let's fix that." Disagree with their cause, or predict that they'll fail, and we can converse sensibly about why they're wrong. Making fun of them for falling into the common psychological modes of religion, and I think you're missing the big human-condition-shaped elephant in the room that simply isn't going to go away.
But there's another issue here. Not only they believe they can escape not only the fate but the cause of the human condition, they are also, in many ways, blind to it, and this is what makes them different from other religions and even Communism. They are privileged, blind to the misery of the under-privileged, and their persecution story (every religion has to have one) revolves around intellectual superiority. In fact, they are distinguished by having led lives somewhat removed from the concerns of most people. So I think that makes for a pretty lousy religion.
The big question is, are they harmless?
> A guy wept because previous generations didn't have Kahneman and Tversky's foundational findings in behavioral economics? Excuse the following several minutes of nonstop giggling.
I laughed when I read that, too. That story, as given in the magazine, is meant to echo Christians lamenting those generations that lived before the time of Christ and therefore -- by definition -- could not have been saved.
On the one hand, human-equivalent AI is undoubtedly possible, as is greater-than-human AI, as is (to my knowledge: I have some ideas on how to do it) self-improving AI.
And yes, the "blessing of abstraction" effect of transfer learning does get us out of the Curse of Dimensionality somewhat, allowing the sample complexity for general learners to be a bunch lower than the sum of the sample complexities for a series of individual narrower learners.
And, at its very mathematical fundamentals, statistical learning works, in a way that pre-Winter AI just fucking didn't. I mean, theoretically you could learn a super-duper god-AI in a sufficiently large two-layer neural network... it would just be infeasibly large and intractable to train.
But the only way you get a "hostile Singularity" is if your super-duper god-AI isn't just smart, or smarter than one human, or even smarter than one clever human, but in fact more hierarchically general (able to learn more from the same data), quicker-thinking (able to sample and infer faster), and more accurate (able to approximate intractable distributions more closely from the same amount of computation spent on inference) than whole organized teams of trained, expert humans. And then you have to feed it sufficient training data for it to understand the world more deeply than said teams!
Or unless it just bribes or otherwise corrupts said teams (which is, admittedly, the most likely way for this scenario to happen: humans are definitely easier to buy off than any artificial agent anyone would bother to design).
So a "soft take-off" could happen. But I'm really not sure anyone other than ideological "singulatarians" is putting enough effort into causing it for it to be very probable.
Hard take-offs are very unlikely for reasons of sample and computational complexity: educating a human takes approximately 20 years. Even assuming you can go 100x faster with your AI... 20365/100 = 203.65 days, approximately 4 months of Terminator training in which the designer might notice the AI was "evil" and turn it off.
http://harpers.org/archive/2015/01/come-with-us-if-you-want-...
This is what happens when so much power is held by people who are -- how should I say it -- left of average on the autism spectrum and with so little knowledge of politics and history: you get dystopian notions such as meritocracy[1] being touted as ideals, and sad-crazy-funny-dangerous ideas such as those mentioned in this article get funded. But I think some powerful people have always been susceptible to crazy notions or to people peddling them (see Scientology or Rasputin).
[1]: http://www.theguardian.com/politics/2001/jun/29/comment
I think more will read it seriously than not.