Could AI turn on us, or is natural stupidity a greater threat to humanity? | Hacker News Reader

Could AI turn on us, or is natural stupidity a greater threat to humanity?

theguardian.com

Quote:

The philosopher Nick Bostrom, who heads the Future of Humanity Institute at the University of Oxford, says that humans trying to build AI are “like children playing with a bomb”, and that the prospect of machine sentience is a greater threat to humanity than global heating. His 2014 book Superintelligence is seminal. A real AI, it suggests, might secretly manufacture nerve gas or nanobots to destroy its inferior, meat-based makers. Or it might just keep us in a planetary zoo while it gets on with whatever its real business is.

Lol, how do you head an institute at Oxford while being so clueless about the relevant subject matter? This stuff is good emotion-grabbing speculation, but it has no bearing on developments in the real-world field of artificial intelligence. Replace "AI" with "fancy regression" and see how serious these claims sound.

To be clear, there are serious ethical concerns and threats to human life associated with modern AI, but they have to do with technology shifting the balances of power between groups of humans (country vs. country, in-group vs. out-group), not superintelligent Terminators hiding nerve gas in your shoes.

That's what you get when you listen to philosophers. They seem to have a dislike of evidence for backing up their claims

That's probably a bit broad - there are good philosophers doing good work in the world, this just might not be one of those cases.

As far as I can tell, the children are playing with a box that is painted to look like a bomb. Machine learning is not real AI. But that's where all the money is going.

Researchers and engineers are by and large trying to solve concrete real-world problems and don't care what label the public puts on them. Media darlings and lay people are the ones concerned with whether something is "real" AI or not.

Some sibling comments either are not familiar with the book or are mixing up today's AI activities with the goal of general AI.

Bostrom is talking about the results of efforts to create super intelligence / general artificial intelligence (GAI).

I see AI being referred to four things these days:

1. Things that professionals would never refer to as AI. I.e. hyped gadgets and software. The word "smart" has been overused this way for a long time.

2. Today's machine learning applied to business or artistic problems. Often using third party tensor flow and gradient optimization libraries.

3. Next year's machine learning. I.e. improvements in library algorithms or creative uses of them: such as improved tensor flow, gradient optimization, novel data, layer or target definitions, etc.

4. Progress toward GAI, which on a day to day basis often looks a lot like 2 or 3, but where the leadership and budget are there for sustained progress wherever it leads.

GAI isn't going to suddenly pop out of a bottle today. But now is the time to be accessing the risks because in the time frame of decades we are going to cross that line (lots of lines really).

Some risks are existential so best overmanaged and overthought than under. Despite the understandable (in the short term) reaction of some that threats will always be fearmongering or hype.

We like to think that the goal of AI research is actual intelligence. But humans are in the loop. Us humans judge projects, so success is a matter of the computer persuading humans that it is intelligent.

We could end up in the situation that computers are put in charge of important things because we have mastered the art of writing computer programs that can trick us into thinking that they are intelligent. Then it turns out that the programs are stupid. Disaster ensues.

The key question is "how clever does a computer program have to be to convince humans that it is intelligent?". That depends on two sub-issues.

First, are our assessments hackable? We might in time stumble over a clever trick that makes computers seem impressive to humans. The glib talkativeness of GPT-3, which impresses every-body, hints at this

Second, how keen are we to fool ourselves and just believe, without all that much prompting from the computer? The article worries me because of all the love given to the narrative in which the computer really is intelligent, impressively so, dangerously so. Erring on the other side, thinking a stupid computer is intelligent, is also a big risk. We seem blind to it.

There is no connection between the questions philosophers ask and any actual technology or even concept that people are currently aware of. It's interesting to think about, but more as a thought experiment than as something to be remotely concerned about.

An observation though:

- from what i know about people, sentience is the opposite of maximizing only one thing, so I don't see the paperclip thing as an issue, ie anything smart enough to run amok in that way would be smart enough not to. This would be more of a concern for some "dumb" self replicating process, like a fire or an Ice-9 type agent.

Reply on news.ycombinator.com