I argue that compression and not memorization is key to emerging behavior.

And the fact that llms learn languages and can switch quite well in-between (even if you just replace single words) is proof of learning meta/high level abstractions.

I sometimes write a German word or describe something when I don't have it on the tip of my tongue