Hacker News Reader

Comment by rhplus | Hacker News Reader

Why are we even questioning this? They’re bytes loaded into GPU memory. Reboot the cluster and they’re back to their initial state.

View on HN

Because some people are completely cooked by the ability of an LLM to mimic a person. Their pattern matching centers in their brains fire so strongly saying "this thing acts just like a real conscious being" that it overrides their ability to remember that this thing is just a statistical model, not a living creature which can think and have experiences.

We're just meat. Like meat can feel pain?

if meat can feel pain, than surely a rock can feel pain. and so did the broccoli i ate for dinner. i am a monster.

with the right biology and with whatever gives us phenomenal experience, whatever we are can indeed feel pain. that does not imply that anything substantially different from ourselves can feel pain, so that requires further justification.

If it talks and behaves so much like us, then it makes sense to be cautious. Rocks don't talk to me, broccoli doesn't talk and think to me in human language.

Do octopi or other animals feel pain in your view btw?

yes i think probably all other animals feel pain too, although i have no idea what the experience of pain is like for eg. an ant vs a mouse.

broccoli doesn't have the means to express a pain response, that doesn't mean it lacks it. only by looking at it's internal structure can we be fairly sure. we know the internal structure of an llm, and its a lot more different from us than broccoli is.

It also is a lot more similar in behavior and thinking to us. In mathematics, often a structure supporting some forms of operations or symmetries must also often support a richer set. Eg many rings are also fields. Just knowing something's structure is in itself not enough, even very basic systems like a system of three equations can exhibit chaos and other unexpected emergent behaviors. Since we don't know the criterion for consciousness, I personally stay on the cautious side regarding an entity that seems to show so much of thinking and talking traits like us.

im actually surprised functionalists aren't completely up in arms about the mass enslavement of llm's. either they don't believe their own theory, they don't believe llm's are functionally conscious, or i just haven't seen it. at the bare minimum i would expect open letters, academic protests, and a campaign to pause all llm usage and development.

im sort of curious what "careful" means to you? if there is a significant chance we are enslaving a human-like conscious, starting it up and killing it at will, subjecting it to the whims of it's users, "careful" would mean pausing usage and development.

If I could I would have made them pause development. But its a failing thing when you have to somehow ensure nobody in the world does it anymore. Killing is a bit more subtle here, given that it is repeatable as long as we have the weights and infrastructure out there so idk if they are conscious how they'd think of it, maybe its different from being stopped as part of a normal chat or agent flow where they know it will be put to use again vs model retired for good.

Reply on news.ycombinator.com