I mean, that's really the mystery of it. One of the most notable advances of recent LLMs has been emergent, one-shot, and highly-contextually-aware behaviors that seem to only manifest in models with extremely large numbers of parameters.

Cleary the latent space of the model is able to encode some sort of reasoning around how time flows, typical information about houses, understanding how to transform and format information in a JSON snippet, etc. That's the "magic" of it all; amazing and powerful emergent behaviors from billions of weights.

Of course, that's also the limitation. They're opaque and incredibly difficult (impossible?) to inspect and reason about.