I don't think ethics involves consciousness either; you can't inspect other people's consciousness, so how can you put ethical weight on it?

You can assume that other people are conscious like you, and so you put ethical value on that. But then you're really putting ethical value on "being like me", irrespective of whether other people are or are not conscious.