Hacker News Reader

Comment by cmapes | Hacker News Reader

I think it becomes unethical once we see strong evidence that the system could be self-aware. Before that, it's just a simulation. Simulating storm activity doesn't make it rain.

View on HN

It isn't a big leap to imagine a debate between aliens about whether the hairless apes on the third rock from Sol should be considered "sentient", or merely clever animals.

I'd rather err far on the side of assuming sentience and personhood if there is a shadow of a doubt, simulated or otherwise.

Why would they debate? It's very easy to test for [sentience or fake sentience programmed by a sentient]. Unless the aliens think we're robots they can do a mirror test or something.

But such a test does not work in a possibly-tainted environment. A neural net that seems to communicate might just have a hidden encoding of ELIZA.

You can err on the side of assuming sentience if you want, but don't think it's out of self-preservation-empathy-logic.

> It's [sentience is] very easy to test for.

Isn't that only true if you assume that our definition of sentience is universal or that there is no higher development of consciousness? It seems to me that it doesn't even require a very skilled science fiction/fantasy writer to develop such a concept or ability.

As long as you assume whatever you're testing moves around or communicates instead of constantly naval-gazing, I'd bet even crazy-alien intelligence can be seen to be intelligent.

Sure there might be higher levels, but we're not talking about higher levels?

You were explicitly talking about testing sentience (which does not require "moves around" or "communicates"). Sentience is simply a concept we have come up with to differentiate organisms. It wouldn't seem surprising to me if something more evolved used a different concept more appropriate to their physiology or philosophy. Hence I think you need to be careful if we are about to apply our concepts to classify something.

They might have a different standard of important but that has nothing to do with checking for sentience. If they're so advanced beyond that the original analogy breaks and while I might have that discussion at some point I don't think it fits here.

On the topic of sentience, I just think it's exceedingly unlikely for there to be an intelligent organism that does literally nothing with its intelligence. Some kind of self-aware rock. And on top of that the mechanism of the intelligence would have to be almost impossible to study or plug wires into to try to force it to communicate.

We choose the definition of sentience so that it includes us. There's no reason to believe that an alien species might not do the same, and that we might not make the cut. Hell, some of the worst racist rhetoric on the planet involves seeing other humans as non-sentient.

Why did you link a story where the ability to think/feel was clearly established in a bafflingly-different species to argue against me?

It's not like this is some crazy threshold designed specifically to apply to humans, it applies to a bunch of animals too.

At a certain level it's kind of like checking for turing completeness.

The whole point is that we're blind to our own biases, as fish are blind to water. That includes our bias towards physical matter over simulated matter. (It's funny how the notion of a simulated universe unsettles people and destabilizes their narratives, despite the fact that discovering it would change precisely nothing about the subjective reality of our universe.)

Reply on news.ycombinator.com