"Well, what if they took all the AI power that makes that GO thing beat the best player, and converted it into regular intelligence? It would probably be as smart as the average person!"
Is a real question I've been asked several times by friends. Of course, no one wants to be friends with "that guy" at a party or family dinner, so I usually just say "Yeah, that would be neat" and move on. But I really do like talking having serious conversations speculative AI and geeking out over AI. It's hard though, because most people just want to talk about the kinds of AI they've seen in movies, which are unrealistic compared to what's possible and practical.
I tell them that any AI that's designed to only solve one type of problem (or a finite set of problems) is "weak." An AI that can figure out how to solve an arbitrary problem is "strong."
I then tell them that we've never been able to create a strong AI, and we've never been able to convert a weak one to a strong one. They ask why not. I tell them that every weak AI, at its core, has a mathematical way to evaluate its outcome. Every time it does something, it does it by assigning some number to the outcome, and it tries to get that "number" as close to the "good" number as possible. All of our advances in weak AI have been in ways to come up with better numbers, or use computers to come up with good numbers to situations that we understand very well but aren't good at attaching numbers to. Finally, I say that with the way we make weak AIs, to make it into a strong AI, we'd have to come up with a way to assign an accurate number to literally every single decision the computer might be asked to make. But there are infinitely possible decisions, so we'd have to come up with infinite ways to come up with a number. To come up with strong AI, we need some "universal" way for the computer to think about the world, but we have no idea what would look like.
This explanation is hardly adequate for a true understanding of AI, but it works pretty well for helping them understand the gist of the problem without using some misleading metaphor that would just give them another bad understanding.
Quoting from the excellent "Wait But Why?" blog post The AI Revolution: Our Immortality or Extinction (http://waitbutwhy.com/2015/01/artificial-intelligence-revolu...):
> In 2013, Vincent C. Müller and Nick Bostrom conducted a survey that asked hundreds of AI experts at a series of conferences the following question: “For the purposes of this question, assume that human scientific activity continues without major negative disruption. By what year would you see a (10% / 50% / 90%) probability for such [Human-Level Machine Intelligence] to exist?” It asked them to name an optimistic year (one in which they believe there’s a 10% chance we’ll have AGI), a realistic guess (a year they believe there’s a 50% chance of AGI—i.e. after that year they think it’s more likely than not that we’ll have AGI), and a safe guess (the earliest year by which they can say with 90% certainty we’ll have AGI). Gathered together as one data set, here were the results:
> Median optimistic year (10% likelihood): 2022
> Median realistic year (50% likelihood): 2040
> Median pessimistic year (90% likelihood): 2075
> A separate study, conducted recently by author James Barrat at Ben Goertzel’s annual AGI Conference, did away with percentages and simply asked when participants thought AGI [Artificial General Intelligence] would be achieved—by 2030, by 2050, by 2100, after 2100, or never. The results:
> By 2030: 42% of respondents
> By 2050: 25%
> By 2100: 20%
> After 2100: 10%
> Never: 2%
a) What is the computational power required? (stuff figured out back in the 80s turned out to work well for deep learning, but it took another two decades until the power was there at an economical price.)
b) What kind of mistakes do human-like thinking machine make? Additionally, are those mistakes actually a fundamental piece of general intelligence?
Some of the core pieces to general intelligence may be solved soon, but it could take a long time for the economics to become compelling or even possible. Presumably you can collapse time by throwing more resources at an "AI", but humans, with their brains running more or less 24/7 take many years before they can start producing really valuable work. We have assumed you can preload an "AI" with concepts or just copy and paste, but this might magnifying weaknesses just as if every employee at Google was a copy of Sergey Brin.
I except we will start seeing things, very soon, that begins to look very much like general intelligence, but the hardware limits will be an issue.
Remember, the same experts predicted flying cars in the 1960s. We are yet to have self driving cars.
TL;DR: Experts have a poor track record for predicting AI development, they aren't particularly qualitatively different in their predictions from non-experts, and "15-25 years" has been a staple prediction for decades now.
(Statisticians often deride machine learning people for having a poor understanding of statistics).
The point being: You do not need general AI anytime soon, the consequences will still be significant, I would bet.
If we accept that GAI is far off, though, it's very likely that we simply can't have a productive conversation about it at this time. In fact, such conversations could even be counterproductive if we make incorrect assumptions, leading to the illusion of safety.
Most talks about the dangers of AI assume we'll one day finally "break through" and create a metaphorical ghost in the machine, at which point all bets are off. However, this scenario seems awfully unlikely to me. Human intelligence is far beyond that of any other known organism, but there are plenty of intelligent animals on Earth, and the process of solving GAI will almost certainly produce intelligence on par with those animals before it breaks through to human-level intelligence. Dog-level intelligence is a decent milestone; at that point, we'll clearly have created GAI in the sense of creating a being that can exhibit truly "intelligent" behavior (including emotions), but it's clearly not intelligent enough to cause any problems that are specific to GAI. The rebuttal to this argument is that a generalized learning algorithm running on a computer will be far more powerful than a dog's brain, but it's fallacious to assume that the only difference between a dog and a human is the processing powers of their brains. That would imply that a bigger brain (however you choose to measure it) is all that's needed to create more intelligence. As far as we can tell, this isn't true, and architectural differences (which imply "software" differences) seem important for creating higher intelligence.
When we do get to GAI, we don't know if it'll be through software running on a "normal" computer, or if it'll be running on specialized hardware in a specialized shell designed to navigate a real environment, or if it'll be in some other form that we haven't really thought of yet. The dangers and resolutions are very different between those forms, and when we don't seem close to GAI at all, it can be argued that talk about its "dangers" may be a complete waste of time, or perhaps even counterproductive to safety in the long-run.
Why are emotions required for intelligence?
But second, it may very well be true that emotions are necessary for GAI. The GAI probably needs to have some sort of basic instinct or drive; otherwise it'll just exhibit completely random behavior because it has no actual intentions (remember we can't explicitly program intentions for tasks, because then it's not general intelligence). The very fact that the GAI will "want" to act according its instincts/drives implies it needs some concept of satisfaction or at least desire. I think that satisfaction and desire meet most definitions of "emotion," and they can absolutely lead to emotional (as opposed to logical) behavior because any GAI is going to be basing satisfiability and desire off of heuristics, not perfect data, because the instincts/drives need to be general themselves, such that it's impossible to deterministically satisfy them. (Otherwise, the behavior degenerates into something we would recognize as non-intelligent.)
I say "probably" at the start of all of that because we've never made GAI, so it's impossible to know for sure what it takes to get there. However, that model is coherent with evolution and the animal (particularly mammalian) condition. It's unclear exactly what the drives of animals are--easy guesses are survival and reproduction--but we know there are biological responses that heavily guide animal behavior.
For example, we don't eat to live so much as we eat because relief from hunger is enjoyable. However, we've developed an even more abstracted response, and so most of us just kind of like to each in general, even when we're not particularly hungry. When our body can tell we're eating, it releases dopamine, which associates to a mood of well-being. This elevate mood is an emotional response; it's the high-level abstraction we use to choose our behavior.
More complex decisions (involving social situations or whatnot) can just be mapped to ever-higher abstractions that are enforced through mechanisms we developed through evolution. Therefore, it can be argued that literally 100% of high-level decision making is emotional. Even if we ultimately use logic to try to make the best decision, our determination of what we want out of the "best decision" is still emotional. Even more extreme, it may be true that emotion turns out to be the definition of general intelligence, because it's at that point that the agent is making decisions in arbitrary situations based on some sort of internally consistent desire, rather than a prescriptive goal.
So, the idea of a cold drone that can do everything a human can do, but doesn't feel any desire or emotion, may in fact be a contradiction. Achieving behavior that complex may not be possible (in the real world, with the data available in our plane of existence) without some emotional mechanism.
AI improving in those specific tasks are bound to affect people today, it's just the comfy CEO and his pals are not going to be outsourced (or AIsourced or whatever) any time soon, although it might happen eventually.