Because the expectation was too high. If you are aiming for precision, neural networks might not be the best solution for you. That is why generative AI works so well, it doesn’t need to be extremely precise. On the other hand you don't see people use neural networks in system control for cricital processes.
In particular the ability of auto regressive transformer based networks to produce sequences speech while being immutable still shocks me whenever I think about it. Of course, this says as much about what we think of ourselves and other humans as it does about the matrices. I also think that the weather forcasting networks are quite shocking, the compression that they have achieved in modeling the physical system that produces weather is frankly.... wrong... but it obviously does actually work.
While combinations of words are infinite, only some makes sense. So there’s a lot of reccurent patterns there. When you take a huge datasets like most of the internet and digital documents. I would be more surprised if the trained model where incapable of producing correct texts as both the it’s overfitted to the grammar and the lexicon. And I believe it’s overfitted to general conversation patterns.