You have obviously no clue what "theory" means in that context. Playing with LLMs has so far demonstrated very little. Yes, these language models are subjectively impressive, but that's exactly what they have been designed to be. Usefulness is a completely different metric and it will be much harder to demonstrate convincingly.

From a pure programming point of view, the situation is simple: Even if a LLM could translate human language into working code, it just becomes another compiler and the input just another, admittedly novel, kind of programming language.