My predictions:
1. state-space models make transformer based models obsolete
2. Cuda killer gets set loose by AMD
3. The world's first successful head transplant takes place
4. Children of Dune gets greenlit
5. Lex Friedman retires from interviewing
1. state-space models make transformer based models obsolete
2. Cuda killer gets set loose by AMD
3. The world's first successful head transplant takes place
4. Children of Dune gets greenlit
5. Lex Friedman retires from interviewing
We will see whether they work on a large scale pretty soon. I hope they will, but they might not be. There're models which might outperform more advanced models on the smaller scale, and I haven't heard how Mamba performs on GPT scale.
I also think state-space models will make a comeback.