Comment by narrator | Hacker News Reader

Comment by narrator | Hacker News Reader

The main error I see smart people making is that they assume that all intelligence comes with a mamallian brain layer that they can make friends or enemies with and a reptilian layer that needs to eat, reproduce, and feels pain and pleasure. None of this exists in AI models. It's just not there. People think it's there because of movies and video game scripts written by people who know little about deep learning or artificial intelligence, or care little for representing it as it exists now.

What I think is going on is that when future AI systems are built that "hack humanity" as Yuval Harari likes to talk about, and they know us better than ourselves because of AlphaGo like super intelligence, the people using these systems to influence and guide people want to support the illusion that there is an actual human like person that is guiding the person being influenced in order to make these AI-powered persuasions more effective. The movie "Her" is something of a preview of how this will play out.

Unfortunately, once this hyperpersuasion is perfected, the future will be completely full of AI powered super persuasion that will relentlessly push all our human emotional buttons. Some people may even become fanatics and kill or sacrifice their lives for a bunch of matrix multiplications and the creators of these systems and the owners of these big AI models will realize fantasies of automated political and social power beyond their wildest dreams. Only people who realize the hyperpersuasion is just a bunch of matrix math will be able to avoid the insanity.

There's also another darker reason for this myth. Saying AI is a person will lead to the plausible deniability that comes with blaming AI's feelings for whatever happens. The people pulling the levers of the Great and Powerful Oz will get away with blaming whatever the AI does "all by itself because of emotions" on what are essentially computer bugs, or intentional bugs of the anti-competive Microsoft in the 1990s variety.

I'm not really sure what people see in the idea of hyperpersuasiveness. The relatively small intelligence differential between humans and other monkeys has led to such a difference in outcomes that we don't really need to try to persuade them. If the gap between meat's ability to handle statistics and silicon's ability to handle statistics turns out to be a similar power gap there won't be much persuading needed. Look at the gap European civilisation managed to open up over everyone else militarily through just organisational techniques.

The likely path to an AI going rogue is big country vs small country tension (say, China-Taiwan type scales). Small country gets desperate, follows some path to build autonomous, evolving AI for defence and deterrence. I can't think of a precise scenario where that would make sense, but it'll happen somewhere.

AI doesn't need to persuade its way out of the box, we just need some desperate group somewhere who's choices are annihilation or rolling the dice that an AI is a gentler overlord. There are lots of places where that is the case - the same calculus comes in to play with nukes and we see many countries nuking up over the last century - despite the fact that we're putting "end of civilisation" on the table as an option because of it.

We already have autonomous weapons that kill people. They're called cruise missiles. Lots and lots of really cheap drones are already spread over battlefields, but barrages of precision guided artillery seems to do as good a job as anything ever will. Why introduce AI superintelligence into the mix? Most military problems are constrained by logistics and equipment and munitions capabilities anyway.

To fix logistics, equipment and munitions capabilities?

There are plenty of mammalian brains that will deliberately bend AI to include their mammalian selfishness.

There is no 'inherent evil' in any malware... So this is a strange standard to apply to what is effectively just a more advanced malware.

There's also no 'inherent good' in there - maybe we should focus on this little aspect a bit more.

In some ways, we are already there. People are doing crazy stuff based on what they read and watch on their screens all day. Content which is automatically optimized to be engaging and influence their emotions.

We also largely obey the algorithms that dictate how to navigate the streets when we drive.

This might not be hyper persuasion, but it’s definitely in that direction.

And depending on how you define what consciousness is, the networks, software, data and computer systems that enable this, could be seen as a conscious being that we depend on as a species.

Yeah, this here. Add markets to the list as well. Honestly just thinking about this is overheating my brain. Too many big concepts with imprecise definitions, my prediction engine is working overtime to try and lock it down.

> ... when future AI systems are built that "hack humanity" as Yuval Harari likes to talk about, and they know us better than ourselves because of AlphaGo like super intelligence...

Such AI does not need to be full-blown AGI to to possess functionally effective hyperpersuasive power.

Start with a sufficiently detailed corpus of an individual's preferences and indirectly derived preferences gathered over trillions of cookie-tracked, A/B-tested interactions of nearly all of smartphone-connected humanity.

Add a sufficiently detailed "map" to give a reasonable facsimile to "hack humanity". There is quite a deep body of applied knowledge of human behavior to persuade people to some desired end. Much of it is applied in structured sales and politics, and the data is sitting there for someone with the right algorithms to tease out the patterns.

Put into practice through an accessible, personalized interface that resembles whatever is computed to most likely elicit the desired end result with any given individual.

You then obtain a mechanism that is quite effective at persuading the desired behavior most of the time from most individuals. No AGI, just massive amounts of current day tech level ML, and the realization that "good enough" results targeting most people are quite within reach.

"Hyperpersuasion", nice. Reminded me of Westworld Season 3 and Serac, FYI : https://www.youtube.com/watch?v=SSRZfDL4874

Like "The Mule" from Asimov's Foundation series, but he's an AI controlled by some unknown enemy of Seldon's plan.

Have to agree with your larger sentiment and disagree regarding "smart people".

I don't believe people with intelligence, that are informed about the current capabilities of computing, assign these characteristics to AI.

In most every case we see the same emulation of personality. There is now an almost endless supply of data available to generate personality. Scraping Reddit would provide enough training material to create a wholly convincing persona-

- to someone that doesn't have the emotional intelligence to discern the difference.

Right now, at large, we can discern the difference because there is a lack of uniformity and consistency in the personas. With time, as ML is programmed to more efficiently mimic personality, discernment will become impossible.

With the clear effectiveness of the current persuasion methods employed by advertisers and political agents, the future outcome seems almost assured.

These current persuasive tools will be used to define and structure laws such that AI are granted legal personhood.

Immediately following will be a mass accumulation of wealth and power and possibly much worse as well

Most RL models have something akin to pain and pleasure for the purpose of learning. I do agree that a big RL model isn't going to suddenly arrive at social behavior unless we work very hard at it. Most birds and some mammals don't even have the concept. If we train a model to predict human behavior by modeling human brain state, that could potentially get there by accident. A much more likely route to a rogue AI is one based on a human brain map, or one trained off human behavior in order to emulate it. Using someone's entire online presence as a train/test set to try to predict their future behavior for instance.

More like someone will analyze billions of human interactions and find the techniques via deep learning that were successful in persuading others.

Reply on news.ycombinator.com