It's a very funny read.
"See, LLMs that are allowed to use Python perform better than ones that aren't, and Python is symbolic, so I was right all along!"
Looks like a surrender to me.
"See, LLMs that are allowed to use Python perform better than ones that aren't, and Python is symbolic, so I was right all along!"
Looks like a surrender to me.
Isn't his argument that leaders in the AI/ML space have consistently dismissed the need for that the entire point of the article? And that seems like a valid question to be after reading it.
And huge financial implications for the industry.
If he claims that giving LLMs a Python interpreter is a huge win for his paradigm, then major AI companies have been "winning" since 2022.
Turns out that LLMs find bicycles useful too.
OpenAI demonstrated Codex - a version of GPT-3 which could write Python and JS code - in 2021, only a year after the first GPT-3 release.
Here's a demo of Codex doing MS Word tasks using Python code: https://www.youtube.com/watch?v=-Dpl2awseZU
Live coding demo: https://www.youtube.com/watch?v=SGUCcjHTmGY
There's also a demo of doing some data processing using Codex.
The idea of using LLM to write code is rater obvious, and many people talked about it around GPT-3 or even GPT-2 release. People know that many logic tasks require search and it's rather silly to use LLM to do that. If it can generate any text it can generate Prolog code or a specification for SAT solver, and then dedicated efficient tool can handle the computation.
Gary Marcus is so detached from actual research that he might have genuinely missed the 2021 version of Codex. So he might truly believe that they added Python interpreter "quietly", and it's not the big announcement of 2021.
This guy is known as a clown in the industry, and for a very good reason... If he was an actual researcher he would have jumped on the opportunity to make an actual neurosymbolic demos when first language models were released - you don't actually need to wait GPT-3 to do that. But he preferred to relish in writing books claiming that the entire industry is wrong instead of actually building something.