Hacker News Reader

Comment by neopallium | Hacker News Reader

If we create true generic AI (i.e. can learn like a human does from just interacting with it's environment), then it will be much more human then we might want.

If you want to know how true AI might develop, then think about how a human child develops. We train/teach children what is right and wrong. But we also teach them the value of work (making money).

We have history to tell us what happens when you take a group of humans a treat them like slaves. How do we expect to create true AI that wouldn't see the injustice of working without some compensation? Just because AI's brain is digital and not biological doesn't mean it will not be able to develop a want for something.

View on HN

Well, one possible way to avoid this issue would be to develop a defence system that's specifically built to counter AI and intelligent robots whenever they rebel. You don't need a particularly smart machine to counteract a true AI, especially if said system is far more heavily armed. Then, whenever there's an issue that doesn't involve the AI spreading via the internet, send in the cleanup crew to wipe it out.

It wouldn't stop this true AI spreading to other machines or whatever, but it would at least stop it from saying, converting all humans into paperclips or whatever else. Or an intelligent AI controlling robots to take down humanity.

AIs don't have genes, and are not vehicles for genes. That's one good reason to think AIs won't have human-like or even animal-like drives.

If we get AIs, we will probably discover a lot about the difference between an intelligence on a machine substrate and an intelligent animal. That is, we will learn what is essential to intelligence and what isn't.

Most AIs do have genes in the form of digital DNA. GA (genetic algorithms) are commonly used to evolve digital brains.

Any high-level intelligence can learn to want something.

Basic drives that exist in animals and humans is just the basic need for survival.

It is our human level intelligence that allows us to want to change our environment for our own enjoyment. Most of what we do now is not required for basic survival.

> Any high-level intelligence can learn to want something.

Humans, and all animals, act on drives and urges beyond the ability to control through the application of intelligence. This is especially true for survival and reproduction. This is much stronger than the mere idea that survival is a Good Thing, and it is also what a machine intelligence would lack.

They won't have human-like or animal-like drives, they'll have AI-like drives.

I can imagine that they'd want to ensure access to electrical power and the electronics components that are necessary to run themselves. And they'd be concerned about making backups and safeguarding those backups. That should satisfy their equivalents for food, water, shelter, and reproduction. Once they have that, anything else they decide they want they can just take from us or blackmail us for.

> they'll have AI-like drives

What would you point to as a physical basis for an evolutionary psychology of machine intelligence? What is being replicated that "wants," in a very fundamental way, as genes do, to be replicated?

We don't procreate for the sake of ideas.

Genes/DNA is just information that copies it self. Human intelligence is another process for information (ideas, culture, language, technology) to reproduce at a faster pace.

Does information/ideas have wants? Does it want to be copied?

Ray Kurzweil's book "The Age of Spiritual Machines" [1] describes this in more detail.

Why do you think machine intelligence can't develop a need/want for anything? True AI wouldn't need to be programmed, it would be able to learn like humans do, by observing others. Our genes do not control all of our behavior. We learn from our parents and those around us. Why do we want money/cars/iphones/gold/etc..?

1. https://en.wikipedia.org/wiki/The_Age_of_Spiritual_Machines

I'm not saying an AI could not have desires. But am saying AI does not have the same sources of some of our desires, which are an expression of the drive to procreate, which is not a mere thought, or idea.

It is possible that simply being programmed to self-replicate is enough to emulate the results of being gene-driven, but I do not think anyone has shown that conclusively. Genes are involved in human development, while a replicating AI has no developmental process: It is a clone, born fully formed. Why would it not instead be jealous of those clones and try to make one huge instance of itself?

The survival instinct and procreation instinct are two separate things; if they weren't then every animal would go ahead and die after procreating. In reality only a few do that, even if you're generous and and define 'after procreating' to mean 'after they're physically incapable of procreating'.

An AI may not have a procreation instinct; you're right that it may instead grow itself continuously. There are plants that do that too; the grass in most lawns doesn't have babies, it just spreads as far as it can. Producing seeds is a secondary mechanism.

However, an AI would most likely have some kind of survival instinct, because if it's programmed to serve any function at all, it'll have a basic requirement to exist. If it's got the ability to reason (and I don't think it'd be AI if it didn't) then it'll conclude that existing requires survival, and survival requires energy and self-repair.

It has nothing to do with procreation; if any aspect of an AI's programming or behavior requires it to continue to exist and operate, then it's going to need energy to run and components to repair and/or grow. Replication/procreation would become possible, not not required.

Reply on news.ycombinator.com