Hacker News Reader

Comment by IvyMike | Hacker News Reader

At what point does this become unethical (if ever)?

Before you answer, please consider how certain you are that you are not a simulation.

Edit: I don't know the answer, and there probably is a large gray area. But I do know this starts to make me uncomfortable the more accurate it gets. Time to go reread Egan's Axiomatic again, I guess.

View on HN

Even if you are a simulation, would you prefer not to exist? What, really, is the difference between reality and a perfectly-executed simulation? It's hard to condemn one without condemning the other.

If I'm a simulation, I'd prefer to exist and have a legal right to continue doing so.

If you were a simulation, you might not have free will. To even propose that you'd have a decision beyond state-machine-dynamics is a world away.

You might not have free will as a meat brain.

I've yet to see a convincing definition of "free will" that makes any sense as anything other than an abstraction on top of determinism or randomness. To propose that we have any ability to make decisions beyond state-machine-dynamics possibly with some randomness sprinkled on top, is totally nonsensical to me.

Precisely. It's almost certain that we don't have free will. I would define free will as the mere appearance that thought processes are governed by self in a non-local time frame. I would gander that a simulated brain might have less of this illusory 'free will.'

I think the GP is talking about current simulations, not future ones. Current simulations don't have sensory organs, and I doubt the simulated neurons accurately reflect a real brain at this stage. Most likely, early simulated minds would be insane and in great pain, with no way to communicate.

Considering we experiment on animals to reduce human suffering (heck, we kill and eat them just because they're tasty), running a tiny fraction of a simulated brain for a few subjective seconds doesn't worry me much. Of course, as computers get faster and cheaper this could become a huge issue.

It's only tangentially-related, but Robin Hanson has some good talks and papers about emulated minds. The shortest summary of his views is probably this talk: http://www.youtube.com/watch?v=9qcIsjrHENU. A more in-depth version of that talk is at http://vimeo.com/9508131.

> early simulated minds would be insane and in great pain

Insane maybe, but why in pain? Why would you add pain signals without also communication?

Even currently existing lower animals (lobsters and down) don't feel pain - why would you expect a simulation to do so?

Exactly.

If we are in a simulation, then ipso facto, we have no idea what 'reality' outside our simulation is actually like. We can only surmise that it probably resembles our simulation, in as much as the creators probably used their own reality as a template.

Even more reason why there is no functional difference, subjectively speaking, whether we're living in a simulation or in actual reality.

Only when given the choice to enter or leave a simulation does the distinction becomes meaningful.

I think it becomes unethical once we see strong evidence that the system could be self-aware. Before that, it's just a simulation. Simulating storm activity doesn't make it rain.

It isn't a big leap to imagine a debate between aliens about whether the hairless apes on the third rock from Sol should be considered "sentient", or merely clever animals.

I'd rather err far on the side of assuming sentience and personhood if there is a shadow of a doubt, simulated or otherwise.

Why would they debate? It's very easy to test for [sentience or fake sentience programmed by a sentient]. Unless the aliens think we're robots they can do a mirror test or something.

But such a test does not work in a possibly-tainted environment. A neural net that seems to communicate might just have a hidden encoding of ELIZA.

You can err on the side of assuming sentience if you want, but don't think it's out of self-preservation-empathy-logic.

> It's [sentience is] very easy to test for.

Isn't that only true if you assume that our definition of sentience is universal or that there is no higher development of consciousness? It seems to me that it doesn't even require a very skilled science fiction/fantasy writer to develop such a concept or ability.

As long as you assume whatever you're testing moves around or communicates instead of constantly naval-gazing, I'd bet even crazy-alien intelligence can be seen to be intelligent.

Sure there might be higher levels, but we're not talking about higher levels?

You were explicitly talking about testing sentience (which does not require "moves around" or "communicates"). Sentience is simply a concept we have come up with to differentiate organisms. It wouldn't seem surprising to me if something more evolved used a different concept more appropriate to their physiology or philosophy. Hence I think you need to be careful if we are about to apply our concepts to classify something.

They might have a different standard of important but that has nothing to do with checking for sentience. If they're so advanced beyond that the original analogy breaks and while I might have that discussion at some point I don't think it fits here.

On the topic of sentience, I just think it's exceedingly unlikely for there to be an intelligent organism that does literally nothing with its intelligence. Some kind of self-aware rock. And on top of that the mechanism of the intelligence would have to be almost impossible to study or plug wires into to try to force it to communicate.

We choose the definition of sentience so that it includes us. There's no reason to believe that an alien species might not do the same, and that we might not make the cut. Hell, some of the worst racist rhetoric on the planet involves seeing other humans as non-sentient.

Why did you link a story where the ability to think/feel was clearly established in a bafflingly-different species to argue against me?

It's not like this is some crazy threshold designed specifically to apply to humans, it applies to a bunch of animals too.

At a certain level it's kind of like checking for turing completeness.

The whole point is that we're blind to our own biases, as fish are blind to water. That includes our bias towards physical matter over simulated matter. (It's funny how the notion of a simulated universe unsettles people and destabilizes their narratives, despite the fact that discovering it would change precisely nothing about the subjective reality of our universe.)

Given we are comfortable with firing bullets into each other. (i.e. killing what is already accepted as human life)

I find it difficult to see how this could ever be unethical (given the status quo).

Without having an abortion debate, what's actually happening in the simulation is just a random interconnection of very primitive nodes at this point. The network has no refined architecture and, at best, some kind of spontaneous activity. In the number of nodes, it could be (generously) compared to flipping on a cat for one second. But a cat's brain obviously has extremely fine architecture, so the better analogy would be that they're flipping on a completely scrambled cat for one second.

When it escapes and rampages through the city, consuming the living brains of innocent bystanders.

Reply on news.ycombinator.com