- Then the limits on the amount of curatable data available made the performance gains level off. So they started generating data and that pushed the nose up again.
- Eventually, even with generated data, gains flattened out. So they started increasing inference time. They have now proven that this improves the performance quite a bit.
It's always been a series of S-curves and we have always (sooner or later) innovated to the next level.
Marcus has always been a mouth just trying to take down neural networks.
Someday we will move on from LLMs, large multimodal models, transformers, maybe even neural networks, in order to add new levels and types of intelligence.
But Marcus's mouth will never stop yapping about how it won't work.
I think we are now at the point where we can literally build a digital twin video avatar to handily win a debate with Marcus, and he will continue to deny that any of it really works.