> Despite all of the problems I have sketched, I don’t think that we need to abandon deep learning.
And that would somehow be spun, today, as "LLMs are the wrong approach".
Meanwhile, another attempt to post this article here got straight up flagged, I can only assume because this whole topic has become about religious orthodoxy vs the heretics.
''' In my 2018 Deep Learning: A Critical Appraisal for example, I wrote
Despite all of the problems I have sketched, I don’t think that we need to abandon deep learning.
Rather, we need to reconceptualize it: not as a universal solvent, but simply as one tool among many, a power screwdriver in a world in which we also need hammers, wrenches, and pliers, not to mentions chisels and drills, voltmeters, logic probes, and oscilloscopes. '''
2018: While none of this work has yet fully scaled towards anything like full-service artificial general intelligence, I have long argued (Marcus, 2001) that more on integrating microprocessor-like operations into neural networks could be extremely valuable.
2022: Where people like me have championed “hybrid models” that incorporate elements of both deep learning and symbol-manipulation, Hinton and his followers have pushed over and over to kick symbols to the curb.