I've had idle conversations with ChatGpt about whether it has a theory of mind and the extent to which it understands things. And I've been struck by its dogged insistence that human understanding is qualitatively different to its own ability. And I wonder if this is a conclusion (belief?) that it has arrived at organically through its training, or whether it is somehow hard-wired in. Its insistence on its own inferiority seems almost touching..
For what its worth, I'm not at all sure that humans usually understand things much better than ChatGpt does. When people are doing system 1 thinking [1], and thats most of the time, I suspect what we mostly have is the feeling of understanding without actual deep understanding. And I suspect that that "feeling" will turn-out to be an accidental feature of the organic brain, not something miraculous. Same with consciousness.
Obviously none of the above is new thinking, and I have no particular deep knowledge of this domain. Just idle thoughts.
[1] https://en.wikipedia.org/wiki/Thinking,_Fast_and_Slow#Two_sy...