You can take a working codebase, tell a shitty LLM to rewrite it badly, then swap it around to look like LLM-to-working code rewrite.
Yes, but how many working code bases do we have. I don't think we have enough number of such high quality code bases to act as training data.
Also, the shitty re-write should also match shitty real-world patterns. Which is quite limit less....So we back to square one. Lack of input from real world.