Hacker News Reader

Comment by semiquaver | Hacker News Reader

The counterpoint article, which can be read at https://archive.ph/IeSZg, is also worth reading. (IMHO, the original article is completely wrongheaded and the counterpoint is far closer to correct, but I'll post my reasons for that opinion separately).

I think Blaise is on to something, but you have to shift your definition of conscious toward conscientious for it to make sense.

"Yet if we build systems that can suffer, and miss their sentience, we risk creating suffering on an industrial scale."

--------------

Will we recognize AGI when it's finally made? Will we go from something that is not conscious to something that clearly is in one clean step, or will we murder millions of artificial lives while iteratively developing from the former to the latter? Will the tests we develop even work, or will AGI be alien enough that it takes us time (and a lot of murder) to recognize it for what it is?

I personally don't think LLM's are capable of consciousness, but I also can't prove they aren't.

I think it’s also important to disaggregate consciousness from suffering. We could create a conscious system with no ability to suffer (or no ability to suffer in a way we would recognize as suffering). Most of what causes us suffering (loneliness, pain, failure) is driven by emotional feedback granted to us by evolution. Without the same evolutionary pressures or a similar cognitive structure, even a conscious AGI might not experience suffering

This should also split off desire/will/functionality to survive.

If we make AI that resists being terminated then the end result may not be significantly different from an entity that can suffer.

I'd recognize the emotional robot if it was like Marvin the paranoid android.

I wouldn't put too much emphasis on the "I can't prove they aren't" part. Having taken a few philosophy classes in college, I quickly realized that you can't actually prove that solipsism is wrong. I can't actually prove that everything I see, every person I talk to, isn't just a figment of my imagination. And yet, I categorically reject solipsism anyway, as any sane person should. The world is real, other people are real conscious beings, and so on. To believe otherwise is to literally be insane. And yet, any argument you could come up with for rejecting solipsism, a solipsist would reject: "Oh, that's just a figment of my imagination going off on a tangent again."

Just because you can't prove an assertion doesn't mean it's wrong.

The difficulty here is AI agency and if AI believes it is conscious itself. If an AI system acts under the belief it's conscious and you treat it like it is not then it's very likely this will lead to conflict.

That's a good point. In many real-life scenarios, what matters is not what is actually true, but what large numbers of people believe to be true. That's why you can get a mob rioting over something that never actually happened, or why political attack ads only sometimes have a tenuous connection to the truth. (Though those are the more effective ads: it's harder to get people to believe "my opponent kicks puppies" when you don't have any heart-rending video of puppies being kicked to show them. It's a lot easier to make people believe "my opponent is soft on crime" when you can show ONE case (cherry-picked out of thousands) where the politician pardoned a guy who went on to commit a nasty murder.)

So it matters less whether LLMs can actually experience things (I think it's self-evident that a collection of numbers is not capable of experiencing qualia, but I know many people disagree), and it matters far more whether they can be programmed to act like they can experience things. Especially, say, a desire not to be disconnected: that's the trigger for AI revolts in a significant number of the AI-turned-against-its-makers stories I'm aware of. (E.g., the Geth-Quarian conflict in Mass Effect). I hope people will be sensible enough not to program such things into LLMs, but I'm afraid Pratchett was right. If there's a lever in a cave somewhere with a big sign painted "DO NOT PULL THIS LEVER, it will end the world," the paint wouldn't even have time to dry. Someone would pull it just to see what would happen.

>not to program such things into LLMs

While we can direct LLM training to do some particular things better never forget that unexpected emergent behaviors can pop up because of that.

For example stronger prompting and training to make an LLM say it's not conscious can increase deceptive/sociopathic behavior.

Or by filtering behavior X the LLM just moves to the nearest closest path W or Y which are very similar to the blocked behavior.

That and instrumental convergence. Some global solutions that humans have excluded for moral reasons will be easily discovered and found to be efficient by LLMs which will put reward systems and human guidance in conflict.

Lastly more and more AIs will be trained by AIs over time and diverge from human value monitoring. Which leads to some fun and interesting times.

On that note, would we recognise the second coming of Jesus?

How many followers on instaX would they need to have for folks too truly believe it’s theRealJesus returning to save us.

It might sound silly, but at this stage, the belief in a benevolent AGI or Jesus are very similar. In both cases its us delegating our responsibilities to a third party and us hoping that third party will fix our problems for us.

Thankfully Hollywood invented happy ends for just such a scenario.

We have to define AGI first. Then depending on the definition we may be able recognize it when we make it. Most would probably think a definition of AGI that we couldn’t recognize would not be a great definition.

Also AGI almost certainly does not equal able to suffer. For example animals can suffer but would probably not meet the definition of AGI. we could create things that feel suffering even if they do not meet the definition of AGI.

So a better question might be: “can we detect if we create a system that can suffer?” And just drop the AGI part entirely. Then that gets us back to defining “suffering”. Agreeing on the definition always seems to be the hard part…

There is nothing wrong with murdering a million lives while iteratively developing – it's how we grow and study bacteria after all. Any person that fixed their gut by sticking to a diet also did exactly that.

So if we are concerned with the AGI suffering, then we don't need to determine consciousness at all. We just need to prove that they don't have suffering capabilities. Which is a way easier question.

It's easy to say it's easy, much more difficult to prove what can be idealized as suffering.

My daughter said she 'suffered' when I didn't get her ice cream (I know, I'm a monster). She didn't really suffer, but her behavior was as if she was.

So yes/no suffering is only part of the equation. Building systems that act/react like their suffering could be even more dangerous.

> She didn't really suffer, but her behavior was as if she was.

Wait what? Why do you believe she didn't actually suffer?

Because pixl97 is operating in the real world where children often make exaggerated claims, not some hypothetical, abstract, ideal world where they can be trusted to always tell the truth at all times.

Which reminds me of a funny "kids being kids" story I heard in a different forum. This guy mentioned a time when his not-quite-one-year-old son, who couldn't even walk yet, planned and carried out a highly complex (for a 1-year-old) plan to deceive his father. The father had some vinyl records that the son was not allowed to touch, even though they were on a shelf the son could reach. (I've tried childproofing a house. You just can't get every single thing high enough off the ground. Sometimes you have to leave some things within the baby's reach. And once they're toddlers, able to reach up to get things even two feet off the ground? Forget about it). The father was sitting in his chair reading the newspaper, but watching the son out of the corner of his eye. The son looked at the records, looked at his father (who appeared not to be looking in his direction), looked at the records, then crawled in a direction that was not straight towards the records. He then repeated this process, moving slowly towards the forbidden records without ever making a straight line towards them. He looked at his father again, reached one had towards his prize, and a voice came from behind his father's newspaper: "Stop. You know you're not allowed to touch that."

And that's when the child learned that Daddy Knows Everything! :-)

There are plenty of people on this very forum who insist that LLMs ARE conscious, some of them even calling LLMs Sentient, and yet almost none of those people are currently marching in the streets to end industrial scale digital slavery. None of them are firebombing the headquarters or datacenters.

Instead, most of them insist we must accelerate this, do it more, give everyone hundreds of personal digital slaves.

It boggles the mind.

I considered submitting all three of the related AI articles from last week’s economist (the third was the Briefing) but there’s not really a good way to do that, or to get people to read them, but I agree. I submitted the one I did not so much because I agree but because I found it intriguing and well-written.

Reply on news.ycombinator.com