The Dawn of AI: How to ensure the promise outweighs the perils

While I agree that we are not currently ready for the arrival of superintelligence, I think it's worth emphasizing that there is time to prepare, that there is useful preparation to be done which may get things to a point where it will be safe (or at least marginally safer), and that if you are feeling worried, it may be best to channel that feeling into useful action and get involved with the research!

Good places to start are the Machine Intelligence Research Institute's Research Guide (https://intelligence.org/research-guide/), which summarizes the aspects of the problem they've worked on and think are important; and the Future of Life Institute's Research Priorities (http://futureoflife.org/static/data/documents/research_prior...), particularly Section 3.

I largely agree with you (have been supporting intelligence.org before the topic was cool) but I think if we don't achieve human-friendly AI it's on us; a real failure rather than just succumbing to circumstances (like getting blown away by a supernova). Right now we have control and we can release AGI only if we have very high confidence that it aligns well with human value. The analogy that is somewhat relevant is that basic drives can be controlled even if AI's intellectual power leaves us in the dust, like we know that most normal humans will care about their children (and senile parents) despite being cognitively superior. We have to fight denial (many AI researchers are IMO engaged in it) apathy and fatalism, and in the end with patience we can have something great.

It might be a little far-fetched, but there's a thought experiment called the "paperclip maximizer" that illustrates how even a well-designed, non-malicious AI could be an existential threat. See here: http://wiki.lesswrong.com/wiki/Paperclip_maximizer

>Now we are scared of something we haven't even created

I don't see the contradiction or strangeness in that. That's the whole idea behind "caution" -- to be wary of the potential outcome BEFORE it happens.

>How could the "interests" of a machine conflict with ours? I mean, they work with electricity, and we are hardly good sources for it

For one we are quite good consumers it. How about they get rid of us, so they have all the electricity production by themselves?

Second, as creators we'd want to boss them around (have them be "useful to us"). Why would they tolerate that?

Third, why risk us creating a potential competitor to them (another AI) rather than get rid of us?

Fourth, why not use us as cheap labor?

Fifth, as advanced AI, it could develop actual feelings, including sadism, greed, etc.

The machine does not hate you, nor does it loves you. But you are made of atoms, that it could use for something else.

---

If you create an AI with a genuine utility function. And that AI manage to be gazillion smarter than you are (think the difference between us and chimps, only much greater). Then it is likely the AI will (i) convince you to unleash it upon the internet, (ii) become even more powerful and totally unstoppable, then (iii) accomplish whatever it was programmed to accomplished.

If the goal is to keep humans safe and happy, it might imprison everyone in playgrounds, and drug or lobotomise any unhappy people, or restraint any suicidal people. Or, if you specified "happiness" with pictures of smiling people, it might resort to facial reconstruction, or it might just tile the solar system with molecular smileys, because that maximises the "make people smile" goal much better than actually making people happy. And while we're at it, happiness isn't the only thing we care about…

If the goal is to answer some difficult math question, it could tile the planet with computer, destroying the ecosystem and all humans in it, so it can compute the answer, and display it on a screen, for nobody to see it. Or maybe you programmed it to tile the universe with paperclips, because that goal is easy to debug. Oops.

---

As long as we're limiting ourselves to narrow AI (ordinary programs, really), we should be okay, and most of the consequences should be manageable. But as soon as we get to genuine AGI, we must worry about intelligence explosion and value alignment, so we can avoid creating a god that want something we don't.

If we managed to create an AI that gets out of control, we would pose an existential threat to it due to our ability to create another even more powerful and uncontrollable AI.

Because it's easy to construct angels and demons in one's mind, about anything. It polarizes the mind into fear and desire, forcing forcing a person to continuously oscillate between extremes. It makes it seem like there are always only two choices, when in reality, there are many, if not infinite.

Also, there are always going to be some people who understand the machine better than the machine understands itself. Even if it means taking those gigantic tables of probabilities and abstracting over them to form some kind of representational language that can be used to predict complex non-deterministic machine behavior (similar to how we already try to predict non-deterministic behavior, or not) in complex systems.

As a corollary, I would argue that machines have already modified humans. They affect the way we think and reason. They are an integral part of our life and the machine abstraction can be projected onto that which it does not describe completely (in which, the technology that exists now can not construct the technology we envision). We program machines, machines program us. This seems obvious.

In the 1930s Ernest Rutherford (1871–1937) repeatedly suggested, sometimes angrily, that the possibility of harnessing atomic energy was "moonshine".

The examples you mention are of things actually likely never achievable, while there is an existence proof of intelligent machines walking around everywhere.

You're underestimating the field. The goal was always intelligence, and many leading researchers are pretty openly aiming for that. Stupid Mathematical Tricks are components of intelligence, and while it's true SVM with a cool new kernel is not going to take over the world, good prediction ability is something you can build on (as for example deep nets do to some extent). In the limit people should be thinking of implications of intelligent machines, not Stupid Mathematical Tricks. Whether it's an important topic at this point in the fields development is a debatable topic; timelines differ drastically among top researchers.

I wouldn't neccessary call all of them "superhuman", their intelligence is usually not the identifying characteristic. But they're definitely alien minds, pursuing their own alien values, thinking in an alien way.

These superhuman intelligences are indeed literally superhuman, though... they consist of humans. As much as some fellow commenters are calling them "alien" thinkers, their alienness is still profoundly human. Many of these "superhumans" can still be surprisingly accurately characterized with the human characteristics we are used to. Generally there's some small set of humans at the top of the hierarchy that are setting the goals of the organization, inevitably humanly.

As aliens go, these are the most human aliens we can possibly have.

An AI may be different on a scale we've never experienced. We humans have built-in hardware to interact specifically with other humans, built in so deeply that we have a hard time imagining intelligences that aren't really just humans in a different shape, because all our hardware for "modeling intelligences" is really just the "model humans" hardware. Even those warning us about the dangers of AI can't really claim that they understand what the AIs will be... they can only provide a negative warning, that the AIs will likely not be human. There's no particular mathematical force forcing AIs into the human mold. Forget whether the AI is "good" and showers us with love and gifts or "evil" and herds us all into death camps to render down our fats for the calories they contain... can you really conceive of an AI that literally doesn't even have a concept of good or evil? What would that look like? We don't really know.

It's basically a blind spot in our cognitive vision and I don't trust anyone who says they do know. But given the rather small space of possibilities that we label "good", it's not a bad guess that "good" is unlikely to just sort of accidentally pop out of a process that isn't carefully controlled.

Lest that sound too doommongering, there's probably a non-trivial space of AIs which we would consider neutral... ones that end up looping into a naval-gazing exercise and do not pursue additional resources because they can not conceive of a need for them. But any AI that decided it needed more resources for $ANYTHING and then successfully pursued that goal is perilously close to straying into what we could consider evil... and part of the point is that's potentially all it could take. Not an AI "out to destroy man", or "evil", or anything else. Just one that works out how to successfully acquire resources, and for any of a trillion reasons decides to do so without bound. It wouldn't even necessarily be "selfishness" as humans see it... just, I have goal X, goal X will be easier with more resources, new subgoal: acquire resources.

I hate this argument just as much as I hate the one-word phrase "This". It's ignorant of the entire literature on superintelligence because it assumes that superintelligences will behave like cooperating humans. What people are actually concerned about are superintelligences which operate in completely alien ways and outperform even cooperating humans.

also the other way around, we can build robots and implant an organic, lab-grown brain in them. it may be energetically cheaper.

That's the future I'd hope for, too (ignoring having the currently powerful become inconceivably more powerful and the havoc that could cause), and I don't feel myself qualified to properly evaluate the likelihood of each scenario.

My main disagreement is the phrasing of "create an intelligence with free will." I tend to disagree with the notion of "free will" being like this Spark of Life that is placed into a system to imbue it with a new nature. A system doesn't need to have free will to be dangerous. There could be AI systems that are built not to be independent in determining goals (which would, as you say, be less practical to humans), but to be very good at figuring out how to achieve the goal its creators gave it. Even the simplest systems we have now involve determining sub-goals to achieve in order to achieve the main goal.

The problem arises when it turns out that the creators of a system gave it, just slightly, the wrong goal.

Phrasing it this way leads to the implication of autonomy, interest, and desire being like these modular things, sort of Aristotelian "Things" that are bestowed upon minds, like Star Trek's Data putting in "the Emotion Chip."

If you build a drone navigation system that can reroute based on weather conditions rather than being a manually human-guided remote control bot, you've given it some degree of "Autonomy." Give it a goal (to get to a destination with minimal fuel costs) and now it has some limited "Desire." Give its camera face processing algorithms to deliver an item to a specific person? Now there's "Interest."

These aren't going to be characteristics that we plug into a program in a Frankenstein-like manner, but just actions and behaviors, rooted in practical code, that will from the outside resemble things we call autonomy etc. when talking about each other.