Did ELIZA have it? LLMs are the same: in essence both are text transformation algorithms. One being rule-based, the other stat analysis based.

Having said that, your concern is technically correct: we don't. Because we don't know what conscience is, we cannot provide a definition against which to test whether it conforms or not.

Sort of like when Terminator robots will be exterminating the human race they probably won't be technically killing us. Just, like, reprioritizing world's water from humans to datacenters, all of it. But not killing people, no.

we can't define it, therefore we can't even assess the distance a "being" is from becoming conscious. We assume other humans are sentient because we personally are (or feel we are), but there is no way to guarantee that i'm real and the rest is not mimicking a behaviour.