That seems like a good focus. Why learn details that can change within days of it being released? Instead, train the models to have good general knowledge, and be really good at using tools, and you won't have to re-train models from scratch just because some JS library now has a different API, instead the model goes out to fetch the latest APIs/gossip when needed.
You feed the model approximately all the text you have ever. And some things like 'popular culture of 2025' won't change, just because the calendar changed to 2026. Just like the popular culture of the 1980s is what it was, and won't change.
Even multiplying that to approximately 100% of that corpus plus adding lots of non-internet text, will pale in comparison to all the non-text training data we will (or are) feeding our coming (and existing) multi-modal models.
If I may go out an a limb here: either we will see continuous great progress on text-based LLMs alone, or multi-modal models will become the next big focus. (Or both.)
That's because people are hungry for progress, and going multi-modal is the obvious thing to try to focus on, if text alone proves infeasible to drive progress.
Just to be clear: I make no prediction here on whether multi-modal will lead to progress, just that people will obviously try it and try it hard, if the focus on text starts to stall.
It's not that facts change across time, but the relevancy of the details change. For example, it would be great if we could teach LLMs all the APIs all React versions have ever had, but if we do that for everything, there will be no limit to the weight's weight, and we'd need new weights every quarter if not more often. That seems very unsustainable.
So the information that corresponds to "What is the current React API for X" changes whenever the API changes, but "What is the React v5 API for X" remains the same. Having the model being able to look up those things via external channels would let us use the same models for way longer, if you need "up to date data" about things.
And it's far from sad that we have so many resources, we can give everyone a supercomputer in their pocket just to take selfies and talk about Pokemon. Why would our AIs be any different?
But you're right, shareholders of OpenAI want profit, not progress, and they'll give us our intellectual Big Mac.