My point is more that they won’t need new training data to extrapolate to new problems, especially if that new problem is just a new syntax or API. Put the whole shebang into the context. Done. Yes, you need super long context for this, or just pretty good search over it, but I think this is likely to become a solved problem.

I think it case of coding it may not be as bad, because new training data (AI generated code) is always empirically validated by tooling and by consumer. It may not be good but it mostly works, otherwise it is discarded or patched, so it has a bottom bar of "it works".