I'm not sure his argument holds that foundational models will no longer function after the bubble pops. There's plenty of open weight models that are competitive which are more likely to exist in a world with abundant cheap GPUs.
Day 1, that's not going to matter much.
Day 30, it's barely going to be noticeable.
Day 365, you're going to be able to tell that you're using an outdated model—whether you're using it for general world-knowledge stuff, or for coding.
Day 1826 (ie, Year 5), they're going to still have some uses, but they'll be so out of date that they'll be actively detrimental in many areas.
>When the AI bubble bursts, there will be stellar bargains on GPUs…
>these standalone models can do amazing things
>The things these open source standalone models can do will only expand, and they will become a given for our computing applications.
I think of the “big” foundation models as the “fossil fuel” of AI. Once the bubble pops and we can’t afford to train any more of them, we’ll be distilling and remixing the ones we managed to make during this weird period where they were feasible.