Do you really think he knows that little? I mean fair enough you've been lecturing on it, but he was lecturing a decade ago, at Stanford. Then he took a little break to you know, run AI at Tesla...
This makes Karpathy look worse, not better.
I just think he puts on very rose tinted glasses when looking to the future rather than seeing the problems hitting ML model design/implementation now. We had a great leap forward with Attention, it woke an entire industry up by giving them something solid to lean on. But it also highlights we should see a _lot_ more pollination of ideas between maths, sciences, stats and comp-sci rather than re-inventing the wheel in every discipline.