LLMs are Markov Chains [1]. "Emergent abilities" of LLMs can be explained by decrease of perplexity in text prediction [2].

  [1] https://arxiv.org/abs/2410.02724
  [2] https://arxiv.org/abs/2304.15004