Without training on real token patterns, what outputs would you even expect to see? It's an LLM. Literally, a model of language. A language is necessary, by definition, even if you have to conjure it within your context window.
You're asking something closer to: "if LLMs can handle ANY language, why not assembly or something low level, close to the metal as possible?" That's a valid question.
My initial response is two-fold:
1) because someone still needs to verify and debug the source code - so human readability still matters.
2) because semantics are king. LLMs work because they can suss out relationships between tokens. At the lowest level, hardware doesn't really operate on tokens - moving 8 bytes from one address to another carries no information about what those 8 bytes represent to the larger program. Higher-level languages allowed humans to form richer semantics, just as they allow LLMs to express richer semantics with fewer tokens.