This applies to any transformer-based architecture including JEPA which tries to make the tokens predict some kind of latent space (in which I've separately heard arguments as to why the two are equivalent, but that's a different discussion.)
It not clear to me how well that maps onto LLMs. Our wetware predates language, and isn't derived from it. Language is built on top. LLMs are derived from language. I think that means that the intermediate layers are very different from the brain neurons, but I don't know. It's eerie how well the former emulates the latter.
Even more interesting, and getting off on a bit of a tangent, there is also a mode that I use for revealing emotions that I don’t have words for (alexythmia): I open up a text editor, stare off into space, and let my fingers type without “observing” the stream of words coming out. I then go back and read what I “wrote” and often end up understanding how I’m feeling much better than I did. It’s weird.
Edit: also, playing with local models through e.g. llama-cpp in “thinking mode” is super fascinating for me. The “thought process” that comes out before the real answer often feels pretty familiar when I reflect on my own inner monologue, although sometimes it’s frustrating for me because I see where their “thinking” went off the rails and want to correct it.