With the preface that I'm not convinced LaMBDA is there based on this article, this did get me wondering how we'd even tell. If we do achieve a chat bot that can convincingly respond in a human way, 100% of the time, how do we tell whether it has achieved sentience or not? How do we tell when mechanistic response ends and self awareness begins?

I mean, we already struggle to tell when animals have self awareness and intelligence. And we're only just beginning to look at plants and wonder. The reality is we absolutely suck at recognizing and communicating with alien (by which I simply mean "non-human") intelligences.

There are a number of species on Earth that experiments really strongly suggest are intelligent, self aware, and have their own languages and cultures. But we only just figured that out in the last few decades with many of them. There are still doubters, and we have figured out how to speak the language of exactly zero of them. We've cobbled together intermediate forms of rudimentary sign language with a few of them, but haven't really achieved communication that goes beyond equivalents to dog training with any of them. But in all of those cases, if we do manage to learn their language, that will pretty clearly answer for us their level of intelligence. Our base assumption will be: if they can talk intelligently, they are intelligent.

With a language model, trained to mimic intelligent human speech we basically have the opposite problem. We have something that can talk intelligently, but might be doing it mechanistically. ...how do we even tell in that case?