It seems like they should be able to “overweight” newer training data. But the risk is the newer training data is going to skew more towards AI slop than older training data.
The OG data came from sites like Stackoverflow. These sites will stop existing once LLMs become better and easier to use. Game over.
No, you need something like Stackoverflow. The crowdsourced ratings system that Stackoverflow has (had?) is the crucial part.