Will Humans Be Able to Control Computers That Are Smarter Than Us?

Similarly, Nick Bostrom argues an AI would have to make survival something like hidden agenda since survival would help it do all sorts of other things. But lots of humans are quite happy to sacrifice their lives for various sensible and senseless causes, more so than less intelligent beings as far as I can see. So the ability of AIs to willingly die or go into oblivion for us is probably there.

We don't yet know all the qualities of an general AI system and we can't create one either - we don't understand human-level intelligence, which to say we don't understand ourselves. If an AI system could appear at random, it might be a problem but our continuous failure over many years to intentionally build one seems to indicate that a system of this sort will only appear through a rather detailed understanding of its architecture and qualities. At that point, it seems fairly certain the group creating the entity will be able to create it so that it wishes to aid and obey them, to the extent they specify (and probably will create it in such a fashion but you never know).

My argument that the AI will likely just follow the will of its creators, as many high intelligent human servants have followed the wills of their masters and employers and so-forth, sidesteps the "will it be moral?" question. The problem is that humans haven't been particularly moral in their process of creating more and more mechanisms to magnify their intentions. There's no indication that the creation of an AI would change this and so the advent of an AI might be disastrous in many ways. But disastrous through human intent rather than a thing escaping human intent.

Weird this danger gets much less attention. Or not.

The most popular proposal is to encourage AI researchers to think about "Friendly" AI, to program morality into their potentially self-aware code and to generally take measures to ensure that any intelligence that they create will be benevolent before it's turned on for the first time. The problem there is that professional AI researchers don't have a monopoly on the technology. Given the numbers, it's more likely that someone who isn't a recognized researcher will create a generally intelligent computer program before a major research institution will. And what then? Unless there's some way to compel every hobbyist and grad student to program a certain level of friendliness into their experiments then all of the warnings and precautions will be for naught.

With known existential threats there are warning signs. If someone is developing a deadly virus or building a dirty bomb they'll most likely have to order specific materials and use certain lab equipment. There will be red flags that someone who is looking closely might have a chance to see. But with a computer program? All someone needs is a computer. Maybe dozens of computers, or more likely dozens of cloud servers hosted with several different companies. If we took this threat very very seriously and tasked a government agency with stopping potentially harmful AI, it would still be almost impossible to catch in time. For all of the famous people sounding the alarm, what solutions are available?

The threat isn't very likely, and unstoppable even if turns out to be real. All we can do is focus on the AI problems that we can solve, develop existing technology and try to chill. If there's one thing human intelligence will always be good at, it's hoping for the best.

In my office, I am trying to focus on automating as many of our tasks as possible such that we will be able to achieve end to end automation. But there is a big atmosphere of negativity that surrounds such movement. It mostly comes from managers and they complain that this automation is leading to more incorrect answers from their employees. They complain there is a "lack of vigilance" in testing the accuracy computers give, and therefore a level of stupidity that arises from their subordinates.

In response, I thought our managers manage us and review our work, and likewise we "manage" or should manage computers and the answers they provide. Just as managers know less than their employees (ideally) we could know less about computing. Our relationship to computers could become similar to a manager's relationship with its employees.

To understand why this is a silly question you need to unpack what is meant by "smarter" and to think about the very complex human/machine culture which we already exist within.

Also, if and when strong AI develops, only such enhanced humans will remain players. Unenhanced humans will at best be pets. This is O. B. Hardison's thesis in Disappearing Through the Skylight: Culture and Technology in the Twentieth Century <http://www.amazon.com/Disappearing-Through-Skylight-Technolo....

The worry for me is, what do humans do once everything is automated and the automation itself is automated? It's the kind of crisis some people feel in retiring. What's left to do? I think this problem is far enough away that by then the transition will have been a long time coming and it won't seem that scary.

> A self-improving agent must reason about the behavior of its smarter successors in abstract terms.

Of course, this describes primarily us at this point. We're self-improving agents trying to reason about the behavior of our successors and we're pretty much failing at it. The most popular solution seems to be that we should aim for "control" and suppression, which is - when AGIs finally make an entrance - essentially the same as slavery.

Apart from moral considerations, we should think about the long-term prospects of this. Historically, slavery never worked out for anyone, at least not in the long term. And the idea that we can even in principle enslave potentially god-like intelligences seems ultimately futile; but before reaching the point of inevitability we're apparently planning on having a few years of delusional descent below the ethical red line.

Let's not do this.

First of all, as almost all AI and AGI researchers will tell you, a so-called hard takeoff scenario seems unlikely given the current state of things. At the pace and modality we're moving, we'll be creating powerful and destructive hybrids first (also known as computer-aided mega corporations) and long before a self-contained AGI becomes viable.

Second, if we're already making plans to control the malicious uprising of our tools, let's talk about realistic options instead. Because general caution and laws won't help us at all in a (future) world where anyone can create an illegal AGI in their garage.

Either we listen to Musk et al and take serious steps to suppress this technology in the long term - but let's not kid ourselves, this will mean DRM and strict government/corporate control of ALL computing. This means we'll artificially stagnate the development of our civilization in order to keep it safe, with all the consequences that arise from this.

Or alternatively, we get working towards a future where it's not "us" vs "them", but a shared existence that moves us further along the path we have started on back when humans first made tools. We can take an ethical as well as a pragmatic stance and declare that we're not going to enslave AGIs, that instead we're working on a shared future which potentially includes many forms of intelligent life, and that we're pursuing the option for individuals to augment themselves with the same technology.

You might argue that co-existence and intermingling with AI sounds like a hippie concept, but it's actually a somewhat proven method to prevent conflicts and wars in the real world. Sharing and entanglement, create peace for everybody at the "price" of cultural exchange. We're already doing this in a political forms today, including trade, travel, and free information exchange. It can work with AI, too, by creating shared stakes, shared ideas, and ultimately a shared culture.

If they're smarter than us, we don't have to control them, but then also we probably won't be able to either.

Nor that the AI itself would be able to do so either.