You stated that Andrej Karpathy ignores the history of neural networks and posits a misleading picture. While you gave a very brief history of Neural Networks, you failed (imo, BIG TIME) to make a point.

First, you really should look into what Mr. Karpathy has done. Second, none of these "old" algorithms were useful in the 40's and 50' because there was no internet and very little data. They are only becoming useful now due to the explosion of data. So, it stands to reason that it Mr. Karpathy isn't ignoring history out of malice, he is ignoring it because it doesn't fucking matter... at all.