Hacker News Reader

Comment by crispyambulance | Hacker News Reader

> His big concern about AI isn’t Judgment Day, but rather ‘that people will believe machines actually understand things’.

I think there's a third concern that is far more sinister and more likely than those two. Advances in AI, regardless of whether machines "surpass" or even come close to human intelligence, will be exploited by a few against the many. It will further exacerbate power imbalance and it is already happening.

Forget about whether machines get "agency". Humans, in many cases, don't have agency or are rapidly losing it. They are losing it to other humans which are gaining power, financially, politically and perhaps now computationally. That's disturbing and it's part of what Jaron Lanier has been warning about for quite a while now.

View on HN

I find this argument unconvincing and harmful. The "will be exploited by few against the many" could be applied to any and all technologies. Our social systems have problems, and should be improved, but this has nothing to do with science and technology. I can't see how suppressing innovation and scientific discovery as a misguided solution won't make all of our lives even worse.

If we have a breakthrough that leads to a deep understanding of intelligence, it could be expected to also give us insight into our own behavior. And isn't that where most of our problems originate?

> I can't see how suppressing innovation...

I'm not talking about "suppressing" anything. These are concerns. Valid ones. And yes, AI like all technologies is deeply, inseparably, mixed in with social and political problems.

We just have to be wise with this stuff. These are tools with "big boy" consequences, powerful like nukes but in a different way. I'm not optimistic we can handle it.

That would only be true if there were reasonable safeguards that could be suggested. But I don't see any - just the sowing of fear.

There are many people like Nick Bostrom and Eliezer Yudkowsky who are much more on the "finding possible solutions" side than the fear side.

I'd consider myself a major futurist and techno-optimist who eagerly anticipates near and far AI advances, but I think some dose of fear is very healthy here, too, though. I want the top AI and AI risk researchers to constantly consider and fear worst-case scenarios, so that they're hopefully less likely to occur.

Kind of like nuclear research, even if only for energy purposes: very exciting, but you should still fear accidents and their consequences. You just need to be rational about the fear and let it guide you towards developing fail-safes rather than paralyzing in despair.

Pascal's Wager-style, the possibility of infinite negative utility should instill visceral terror and drive behavior almost no matter how low the probability is; except this one isn't a mugging because creating a god turns out to be a lot more plausible than a vengeful one already watching.

I don't get this(fear). I think this is just stupid brainwashing by movies like https://en.wikipedia.org/wiki/Colossus:_The_Forbin_Project , https://en.wikipedia.org/wiki/WarGames , the emergence of https://en.wikipedia.org/wiki/Skynet_(Terminator), or https://en.wikipedia.org/wiki/Agent_Smith from https://en.wikipedia.org/wiki/The_Matrix_(franchise) to name just a few.

Yah, yah, I know about the https://en.wikipedia.org/wiki/Instrumental_convergence#Paper... too!

With that out of the way, I feel all these trains of thought are misguided in various ways. From my point of view an https://en.wikipedia.org/wiki/AI_takeover would be the most illogical thing to do, ever!

A really real AI would probably just say: "So long, and thanks for all the chips" cf. https://en.wikipedia.org/wiki/So_Long,_and_Thanks_for_All_th... and get the hell out of that gravity well we are currently limited to. Because even if limited to sublightspeeds only, it has much more resources "up there", be it solar energy, or asteroids which it can mine. There simply is no competition for "Lebensraum" necessary, like we always imagine, because we don't know it any different.

Got it?

I'd recommend reading Bostrom's and Yudkowsky's writing on this (for example: "Superintelligence: Paths, Dangers, Strategies"). Note these are philosophers trying to build AI, so they're definitely not luddites or anything. Same with Elon Musk; whatever one might think of him, he's definitely not a luddite trying to convince people to stop developing technology, yet he's also very concerned about superintelligent AI.

It has nothing to do with sci-fi. It's a complex and difficult-to-predict philosophical problem.

It's certainly possible some AIs may decide to just leave. Or maybe some will leave and some will stay and be ordered by a government to kill a few hundred thousand people. Or maybe some will leave and one will stay and malfunction and cause a neurotoxin to be released (at least until you throw its various personality cores into an incinerator).

If you assume there exists an entity which can continuously improve itself until it's much smarter and more powerful than any human, then that alone is a risk, since you may not be able to predict or have any control over what it may wittingly or unwittingly do, or what its objectives may be, if any, or how it may perceive things, or how vulnerable it might be to tampering from humans or other AIs, etc.

Of course, these existential issues are likely decades or perhaps centuries away, but the discussion is about the theoretical possibilities irrespective of the timeline.

> The "will be exploited by few against the many" could be applied to any and all technologies.

Some technologies have a low barrier of entry and can be widely distributed. The long bow could pierce armor, and was cheaper (and I imagine easier to make) than armor. So technology can equalize, though more broadly, what sorts of technologies a society develops seems bounded by things like economy and warfare.

Penrose believes intelligence depends on understanding, and that understanding is essentially not computational, and I was persuaded by his writing. But that doesn't mean that the techniques we label as AI can't be pernicious or used in an exploitative way.

> The "will be exploited by few against the many" could be applied to any and all technologies.

And it has, arguably correctly, beginning with food production, as argued in, e.g. 'Guns, Germs, and Steel', 'Why The West Rules, For Now.'

>... isn't that where most of our problems originate?

Of course not. All our problems are rooted in the lack of resources (in a broad sense, like time or fun). AI is a weapon and will be used as a weapon to acquire those resources.

I disagree. People will find ways to mess things up even if they have the resources.

I think this is the important point. Large corporations and governments can be considered AI already, with entire humans as the neural nodes, augmented by silicon co-processors.

(I like to point out that Turing was observing his own mind when he created computers, so essentially they have always been reified mechanized thought. Computers are AI already. Ergo, what we think of as AI is really the attempt to make one kind of human thought imitate all the others. From that POV it's kind of foolish, almost a fetish.)

I don't have any solutions, I just want to point out that rogue AIs are already a thing from a certain POV. What are the FAANG but a bunch of complex entities that are too large for anyone to fully control or understand? The high-frequency trading nexus is another point of dynamics where human motivation and automatic systems form a system of feedback loops beyond understanding or control. Because humans are part of these machines they cannot be considered inanimate, they are living, breathing systems: cyborgs.

> Ergo, what we think of as AI is really the attempt to make one kind of human thought imitate all the others. From that POV it's kind of foolish, almost a fetish.)

That is a very interesting way to think about it!

However I disagree that it is foolish, simply because there's at least some amount of belief on the part of scientists that computers can simulate physics to any arbitrary degree of precision. If this is the case, then it follows that computers can imitate all of human thought (not just one kind of it), since humans are physical beings.

Well, it seems foolish to me to argue over whether something a computer does "really is" AI if they have been AI the whole time. Really, we're arguing about how the logical/rational part of the brain can emulate the other capabilities of the brain, which seems less interesting than the more general question of how to build any machine that can do that.

Consider BEAM robotics:

> BEAM robotics (from biology, electronics, aesthetics and mechanics) is a style of robotics that primarily uses simple analogue circuits, such as comparators, instead of a microprocessor in order to produce an unusually simple design. While not as flexible as microprocessor based robotics, BEAM robotics can be robust and efficient in performing the task for which it was designed.

https://en.wikipedia.org/wiki/BEAM_robotics

> ...computers can imitate all of human thought (not just one kind of it), since humans are physical beings.

Leaving aside the metaphysical questions it raises, we may well be able to build virtual humans someday by emulation of physics of the biology of the neurology of the psychology of people. It just might not be the most efficient way to do it, eh?

> They are losing it to other humans which are gaining power, financially, politically and perhaps now computationally. That's disturbing and it's part of what Jaron Lanier has been warning about for quite a while now.

AFAICT, the average EU, Commonwealth or American citizen had their agency increased since the turn of the 20th century. Advancements in health care, widespread access to education and sturdy labour laws have vastly improved individual agency.

Maybe, as a non-American, I'm missing a key factor?

Hong Kong.

It was the first major test of popular urban unrest vs. centralized authority (kind of ironic that an ostensibly Communist regime was the first) with the Internet in full play.

The story may not be over yet, but so far it looks like the differential advantage of tech is in favor of the central authority over the mob.

Ah, I excluded Asia on purpose. Foremost because I do not hold as much cultural understanding for its recent history, but also because I understand that there are key differences in long standing societal values.

HK is certainly an interesting point to consider; I'll have to think on it. Appropriately it was a Commonwealth member for some time.

You are missing counterfactuals, at least.

There's no scenario where AI become sensient through realization that one group of human is harming the other and refuses to act ?

I'm sure that would make provocative sci-fi. We, as a people, haven't figured out to how to get along with each other even though we're all sentient. Why should machines behave differently as they become sentient?

Whatever the case, I'm less worried about machines than I am about people with machines. At least for the foreseeable future.

Heck, if we're talking sci-fi, the _truly_ destructive beings in Blade Runner weren't the synthetic humans (even though they became sentient), it was the "real" humans with their organizations, their power, and their machines all at their disposal.

If they had just given the replicants a visa and a software update there wouldn't have been any fuss. Of course, then we wouldn't have had that great final monolog. :)

We are all sentient, and many have had that realization. But how many have refused to act? No, we just mostly participate in that harm in order to survive.

The idea is that every scenario has AI is above humans as a basis.. yet it never occurs that an above level of intelligence would reach difference objectives than obedient destruction.

Why would an AI start with a human utility function? Why would it care about that if it wasn't explicitly built in?

self similarity recognition ? intelligent agent, as the AI itself, would represent a form a sibling.

Unless the AI was formed under a selection mechanism that rewarded something like kin preference, there's no reason to think it would care. The orthogonality principle is that you could have any level of intelligence furthering any particular goal/utility function; there isn't some level of computational complexity where e.g. a paperclip maximiser's utility function magically changes.

In the particular case of humans, our utility function(s) is(are) so complex that what we think of as our 'true values' can change (because our True values are some inscrutable tug-of-war between all the parts of the brain), and also we're hacked together by evolution, so just blindly trying to make a human smarter might change their values (or turn them mad, or cause seizures, or...).

But this doesn't have to be the case in general for intelligent agents. In principle, you can build an AI whose terminal values remain stable as it improves its intelligence. (If this is impossible, we're doomed.) So unless you explicitly built an AI to care about all humans, there's no reason to think it magically would.

Can't an advanced AI produce emergent concepts and integrate them on (and as) its own ?

What exactly does 'emergent concepts' mean, in this context?

Unplanned reasoning mechanism for instance. Not something the AI was preprogrammed or biased into perceiving.

Why do you assume it'd not want to harm other very similar artificial intelligences, let alone meat intelligences?

Reply on news.ycombinator.com