Hacker News Reader

The problem is AI neither loves us nor hates us; but we're made of atoms that it can use for something else.

So is the meat you eat, yet some people prefer to be vegan.

I've tried beyond meat myself, and it's quite close to the minimum level of quality it would take me (not a vegan) to switch.

Morality matters.

An advanced AI could conclude that biology is wasteful and cruel, and the atoms are better spent making more intelligent machines. It could also be anti-natalist and decide to stop reproduction.

but consent is a very important thing! Why would be AI jump to thus conclusion?

You think consent is important. (This isn't even a umiversal position among humans). Why do you expect an AI to value consent? Do you think praying mantises value consent?

Have you considered some higher features like constent may not be possible until a sufficient intelligence threshold is reached, to allow modelling the mental states of other sentient beings then include their utility (even with a very low weight) in the utility function?

Reply on news.ycombinator.com