For example, the AI models we have now will happily run forever if supplied with external power.
That sounds quite different from my experience of the world!
The AI model doesn’t ever get tired, or bored, or hungry, or anxious, or afraid.
If a model emerged that started procrastinating, say, or telling jokes for fun, or shutting down for long stretches due to depression— then I’d start to consider that maybe it’s got an inner world like mine.