I mean, outside of our own internal experience of qualia, there's no way for us to prove anyone or anything is conscious. But my biggest issue with the debates about consciousness and LLMs is that the "LLMs are conscious" crowd usually has to presuppose a completely materialistic/mechanistic understanding of the universe and of our own consciousness, and that's something that we really don't know for certain. Also, if we were running LLMs on a mechanical or pneumatic or water based computer (albeit very slowly) then the non-electronic computer would be just as "conscious" as the normal electronic computer, and I think it would give a lot of the "LLMs are conscious" crowd pause to think that a bunch of pipes and valves can be conscious. I think there's a lot of magical thinking that arises from the fact that computers are electronic.