In case you are interested, I wrote a piece in defense of understanding in LLMs[1]. The section under the heading "Why think LLMs have understanding at all" is the most relevant.

[1] https://www.reddit.com/r/naturalism/comments/1236vzf