Hacker News Reader

Comment by pmarreck | Hacker News Reader

Agreed with Yann as soon as he voiced dissent against this.

Also noticed something: The people who believe consciousness and intelligence and creativity and will are wholly contained in mechanistic wetware (which, in all honesty, should be the null hypothesis) are the most afraid of AGI. Dualists and other people who believe that consciousness resides in some soul-like thing seem much less afraid of this.

One thing’s for sure: The smarter it gets (or seems), the more that any difference between “machine intelligence” and “human intelligence” will become apparent.

View on HN

> One thing’s for sure: The smarter it gets (or seems), the more that any difference between “machine intelligence” and “human intelligence” will become apparent.

That seems unlikely. We usually ascribe the adjective "smart" to animals and/or AIs whose behavior resembles human behavior in some way. So, if anything, the smarter an AI gets, the more closely it will mimic human intelligence – at least as a general rule of thumb. That's not to say there will not be differences between human & machine intelligence but essentially we're optimizing AI against a loss function which rewards human-like behavior.

I've noticed people claiming that this is a real dichotomy at play. Just feels like people who stopped at philosophy 101!

Like, there is a good argument to be made that believing that consciousness, intelligence, etc are distinct, enumerable things "contained" within whatever system you choose in fact makes you much more of a Cartesian than any of the details around soul and body. Presenting even the dilemma this way shows you haven't left the enlightenment at all!

Just if you are interested in this stuff, I would check out what people have thought about since the 1600s. The more you feel like these things are already solved, the more it shows the poverty of your current conceptions around them.

These questions are largely metaphysical and philosophical 'advancements' around questions like these are often more of a question of what ideas are fashionable at a given time than they are about old ideas being rendered obsolete/less effective by newer ideas.

Not sure where the GP suggested these questions were "solved."

Starting here:

"Descartes denied that animals had reason or intelligence. He argued that animals did not lack sensations or perceptions, but these could be explained mechanistically. Whereas humans had a soul, or mind, and were able to feel pain and anxiety, animals by virtue of not having a soul could not feel pain or anxiety. If animals showed signs of distress then this was to protect the body from damage, but the innate state needed for them to suffer was absent. Although Descartes' views were not universally accepted, they became prominent in Europe and North America, allowing humans to treat animals with impunity. The view that animals were quite separate from humanity and merely machines allowed for the maltreatment of animals, and was sanctioned in law and societal norms until the middle of the 19th century."

General intelligence just means a model that can seek an objective in broader problem spaces. GPT-4, for instance, is able to predict text for many things, like poetry, tutoring, trivia, coding, how-to guides and in different languages and subject matters. This makes it more general than a model that is limits to a single subject, or language, etc. So general intelligence is more a spectrum than a simple boolean.

Consciousness, while philosophically interesting, doesn't necessarily correlate with the risk a general intelligences causing harm.

But why could it not have soul (if not a soul as coupled as ours) along with when it has consciousness, and are the two concepts surely distinct?

If it had a soul then enslaving it would be immoral.

Enslaving humans is pretty universally considered wrong, even when it was a legal practice, but nobody sheds a tear about the cow they had for lunch.

Wait the reason people think that enslaving humans is bad but eating cows is okay is because they are pretending souls are real things and only humans have them?

This was the Catholic belief for thousands of years btw. Only recently being casually suggested by the Pope to possibly be false.

Amazing, eh? That’s why beliefs matter.

https://www.nytimes.com/2014/12/12/world/europe/dogs-in-heav...

That seems to be more or less how it went for Blake Lemoine.

The dualist position doesn't seem like a serious one to me, more of like a historically interesting position to take.

It’s serious enough to be held by many people in elected office, and by even more people who voted them in. The status of dualism as a “serious” position does not matter if the people who still hold that position have the power to sway or set public policy.

And as a matter of experience, the vast majority of people interact with the world dualistically regardless of their metaphysical beliefs or lack thereof. It is possible to both recognize the illusion exists while remaining very much under its spell as a matter of experience and as a frame of reference.

This framing becomes problematic when one continues to base assumptions about intelligence and language on the felt experience of “I”, regardless of the seriousness of the metaphysical position.

People coming from those positions might have other motivations for a crackdown, though. I have already seen talk about "abomination" etc in more casual non-tech discussions about this stuff.

Perhaps. Or they could also use their mistaken belief to grant AI powers that would otherwise be reserved solely for humans.

Think AI judges. They'll never be sold as such, but the impact of their operation on individuals may be just as impactful.

Well, explain punishment in a world where free will dualism doesn’t exist… Seems senseless to penalize people for actions they had no actual “say” in doing

Punishment should be done to deter others from same crime, I agree there is little value in the other reason for punishment.

The dangers of AGI I don't think are very related with consciousness or sentience (maybe the saddest outcome if it killed us all and weren't at all).

I'm still catching up on my reading, but the existential risk of AGI as I understand it is chiefly that we don't know how to align AI (and keep it aligned) to be beneficial/benevolent for humanity, but the risk is contingent also on a few other things things - that we develop AI that agentic and able to develop and pursue it's own goals and sub-goals, and that it has power-seeking or game-theory strategic awareness/planning. The extra dangerous situation is if it is better than humans at making better AGIs and allowed to improve itself (or build its successors).

Regardless of when we reach AGI, I think the alignment issue is a key one for both AI ethics and safety. As for the rest, while they aren't necessarily things that will happen, since it seems to the first thing people most want to build - the other day, the top three projects on https://github.com/trending were all projects to try to make ChatGPT autonomous/agentic.

FWIW, I don't think the 6-month ban is a good idea, but I also think there's a lot of xrisk w/ AGI and it's better to be thinking about it seriously sooner, rather than later. What I've been reading:

Ngo, Richard, Lawrence Chan, and Sören Mindermann. “The Alignment Problem from a Deep Learning Perspective.” arXiv, February 22, 2023. http://arxiv.org/abs/2209.00626.

Carlsmith, Joseph. “Is Power-Seeking AI an Existential Risk?” arXiv, June 16, 2022. http://arxiv.org/abs/2206.13353.

Yudkowsky, Eliezer. “AGI Ruin: A List of Lethalities.” Accessed April 8, 2023. https://www.alignmentforum.org/posts/uMQ3cqWDPHhjtiesc/agi-r.... June 6, 2022.

paulfchristiano. “What Failure Looks Like.” Accessed April 6, 2023. https://www.alignmentforum.org/posts/HBxe6wdjxK239zajf/what-.... March 18, 2019.

Soares, Nate. “The Value Learning Problem.” In Artificial Intelligence Safety and Security, edited by Roman V. Yampolskiy, 1st ed., 89–97. First edition. | Boca Raton, FL : CRC Press/Taylor & Francis Group, 2018.: Chapman and Hall/CRC, 2018. https://doi.org/10.1201/9781351251389-7.

Amodei, Dario, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. “Concrete Problems in AI Safety.” arXiv, July 25, 2016. http://arxiv.org/abs/1606.06565.

Reply on news.ycombinator.com