Relevant reading: https://en.wikipedia.org/wiki/Philosophical_zombie

The answer seems to lie in qualia. But by definition that is something that one cannot authentically communicate. And the notion of qualia itself is increasingly convergent with DNN latent space. Is the redness of red or the warmth of sunlight just a vector in n-dimensional latent space somewhere in our brain computers?

Maybe we're not really conscious.