2. Yes, because the most capable model is the only one that can charge a premium. The rest is a commodity.
They can, but I don't think they will be of sufficient quality. It is a fundamental thing. You can't generate new information from existing information. It has to come from the real world.
Generating training data using existing models will only help the model to capture the exiting patterns more thourly.
ais write 1000 bits of garbage code for 1000 different users one-offs, the users judge that most of those didn't work out but 2 did. Only those good results that humans approved becomes part of the growing body of reference code. The others either get thrown away, or even if they still exist, they are somehow known to be low quality, or at least, not known to be high value.
So an ai wrote some new code that worked and improved the total corpus that other/future ais reference, but it's not just ai output feeding ai input, it's ai output filtered through humans who nixed most of it.
(Maybe exists on github but not used by anyone. It isn't known to be bad, it's just ranked lower, but if nothing else fits as well, it's there to try. That way obscure code that's good eventually becomes known to be good.)
Yes, but how many working code bases do we have. I don't think we have enough number of such high quality code bases to act as training data.
Also, the shitty re-write should also match shitty real-world patterns. Which is quite limit less....So we back to square one. Lack of input from real world.