Two years later, deep learning is still faced with the same basic challenges
garymarcus.substack.com
garymarcus.substack.com
Furthermore, there has been insane progress over the last few years, which presumably would not have happened if everyone had despaired on reading Marcus' 'hitting a wall' article and dropped everything to work on GOFAI.
But let's widen the lens a little bit:
`In 2015, shortly after Hinton joined Google, The Guardian reported that the company was on the verge of “developing algorithms with the capacity for logic, natural conversation and even flirtation.” In November 2020, Hinton told MIT Technology Review that “deep learning is going to be able to do everything.” I seriously doubt it.`
Well, we've certainly got natural conversation and flirtation, along with some kind of logic - especially if you're OK with accepting python programs as a form of response. This is well beyond what was considered possible at the time. We're certainly not at 'everything' but *shrug*. Maybe in ten years.
`nowhere near the ordinary day-to-day intelligence of Rosey the Robot, a science-fiction housekeeper that could not only interpret a wide variety of human requests but safely act on them in real time.`
It exists, it just isn't anywhere near economical for household tasks... https://www.wired.com/story/google-robot-learned-to-take-ord... LLM's look poised to upend robotics.
Marcus then goes extensively into why everyone should be looking at symbol-manipulation systems. And that's fine... But would be much more convincing if he could actually produce something that works. At some point, one asks, if your idea is so good, why haven't you actually solved any problems?
>It’s a kind of glorified cut and paste. Pastiche is putting together things kind of imitating a style. And in some sense, that’s what it’s doing. It’s imitating particular styles, and it’s cutting and pasting a lot of stuff. It’s a little bit more complicated than that. But to a first approximation, that’s what it’s doing is cutting and pasting things.
This argument is terrible because it proves too much. Either AIs are automata with no connection to the world, or if they are connected to the world its just copy-paste.
And somehow infants aren't just copy paste because uuuuh they have "true understanding" whatever that is.
[1] https://www.nytimes.com/2023/01/06/podcasts/transcript-ezra-...
I think the point of the article is that no one has even presented a workable theory for real AI with real comprehension. LLM isn't AI, regardless of the marketing. Until that theory exists, and is implemented, ideas about what AI will someday grow into are ignoring comprehension. That's a required component if you are using the word "intelligence" outside of fiction.
Why not just put a muzzle on it?
LLM isn't AI, regardless of the marketing
That’s intelism. Different models have different capabilities, you can’t decide which is good enough for you to treat as intelligent. You use terms like intelligence and comprehension, which don’t even have a definition better than “I know it when I see it”.
Not even going to discuss that you want to buy an actually intelligent being for picking up remotes.
I didn't advertise a robot picking up a remote for sale as intelligent. I'm advocating that we stop calling it intelligent when it isn't. You're saying it's intelligent.
So either you're ready to offend the previous commenter for their evil advocacy of selling an intelligent robot made to pick up remotes for an advertisement, or you're at odds with your point because you know damn well it's not intelligent.
My guess is the latter.
From the dictionary: the ability to learn or understand or to deal with new or trying situations : REASON also : the skilled use of reason (2) : the ability to apply knowledge to manipulate one's environment or to think abstractly as measured by objective criteria (such as tests)
This robot or ChatGPT, or any LLM theorized to my knowledge does not meet that definition due to the underlying technology not approaching "abstract" or "skilled use of reason."
It's pretty simply not addressing those items. There's no debate. It's a completely different approach to something that has nothing to do with intelligence.
And marketing it as such is frankly a legal issue, in my opinion. I fully expect the lawsuits to continue until marketers are forced to stop using that term entirely.
As an additional note, you've muddied up a lot of concepts here. I'm not sure if you're maybe not a native speaker of English, as that might explain a lot, or if you really accidentally made opposing points sequentially. The latter is a bit funny in a discussion about intelligence. "Intelligent robot" wouldn't refer to one that feels and needs civil rights. The definition is above for your review. It's very much clear, despite your protest, what intelligence means. It is, by definition, not subjective, but objective. And there are absolutely legal definitions with very real legal ramifications.
Also, you should read back through your own comments. You're not a very likable person, and I'm telling you that as a personal favor. You should get some help for this. It's critical for your personal relationships, and it's not very expensive.
I think he's talking about the Google robot that was linked in the root. "Muzzing" a domestic robot would probably mean putting it in a cage, which would defeat the whole point. That's not a solution.
> This is well beyond what was considered possible at the time. We're certainly not at 'everything' but shrug. Maybe in ten years.
This feels a lot like what people were saying about self-driving cars being imminent circa... 2015 or so? [1] The skeptical folks rolled their eyes at suggestions that we'd have self driving cars everywhere in a few years, but lo and behold, they were right. Just because we had a massive amount of progress in the years before, that doesn't mean we were on the cusp of achieving the goal. Turns out going from 99.9% accurate to 99.99% accurate (or whatever the numbers were) is harder than all the believers wanted to admit.
This feels like the same thing all over again. Yes, there's been a lot of progress in LLMs. No, that doesn't mean we're anywhere close to deep learning being able to do 'everything' in 10 years, whatever that means.
[1] https://www.theverge.com/24065447/self-driving-car-autonomou...
True. But Calfornia's DMV makes Waymo publish their stats, and they get maybe 50% better each year. That's a good rate of gain.
Remind me what we're measuring?
"A seasoned San Francisco cab driver might have avoided the intersection of Jackson Street and Grant Avenue, in the heart of the city's Chinatown on the first day of Chinese New Year. An autonomous Waymo robotaxi, by contrast, drove toward the cross streets on Saturday evening, when crowds were blocking all sides and revelers were lighting fireworks, according to two witnesses. Minutes later, some in the crowd attacked the car and set it on fire."
https://www.reuters.com/business/autos-transportation/san-fr...
No, because I have no idea who predicted what at which point in time.
Or yes, because it took [hundreds? thousands?] of years before someone managed to get it to work. We've only worked on LLMs for so long.
i.e. I have no idea how to answer your question, other than to point out the prediction here was not "never", but rather "not in the next 10 years".
What? Where?
> But such projections tend to overlook just how challenging it will be to make a driverless car. [...] It could take decades for the technology to come down in cost, and it might take even longer for it to work safely enough that we trust fully automated vehicles to drive us around.
Do we have self-driving cars? LLM attorneys? LLM radiologists?
No, no and no.
And the continuous failure of self-driving cars should worry you.
At this point I don’t know what that means. I’ll be the first to admit the limitations of statistical language models, but for people like Marcus who have made a name for themselves as skeptics, it’s not in their interests to have falsifiable criticisms.
>The currently enabled Autopilot, Enhanced Autopilot and Full Self-Driving features require active driver supervision and do not make the vehicle autonomous.
Also, how long will you have self-driving? It is notorious for having dangerous regressions after an update. Those AI systems are black boxes with stochastic behavior.
I feel safer, as a cyclist, around Waymos than I do around many human-driven cars. That's even with the cyclist crash last week: it was one of the most common kinds of bicycle-car collisions (bike moving through an intersection hidden behind a truck, hit by a car turning left), and I feel confident that the vehicle responded faster to the situation than a human would have.
From Tesla's own admission:
>The currently enabled Autopilot, Enhanced Autopilot and Full Self-Driving features require active driver supervision and do not make the vehicle autonomous.
Not what was promised.
https://electrek.co/2015/12/21/tesla-ceo-elon-musk-drops-pre...
A hundred billions have been sunk in other self-driving ventures that flopped without much for show. Wonder what could have been made with that money if the hyperhype didn't sink it there.
But if they are getting caught sometimes, that means it is happening undetected in others.
Maybe. Unstructured manipulation is getting better very slowly. But videos such as the Google one and this one [1] are mostly about making the system take instructions in natural language, not doing the actual picking.
Bin-picking of uniform items is a solved problem, and was first working in the 1980s. Bin-picking of random items from a cluttered bin is still a struggle. Amazon has put a fair amount of effort into robotic bin-picking, but it's not yet good enough to deploy.
There's been some good progress on bin-picking recently. Here's Brightpick's system.[2] It has a good vision system which involves LIDAR, and some nice engineering. It's not LLM-based, and it won't do everything, but it can handle most objects that can be picked up by a vacuum picker.
LLMs will always have the same basic challenges - their 'kinda copy what a human has written' algorithm has hit its limit.
He is an advocate of symbolic learning - getting that to work is a lot harder, however it's far more likely to result in genuine intelligence.
It is somewhat embarrassing how many tasks, from customer service to sportswriting to language translation to article summarizing to concept art for films, can be handled by deep learning at its current level of competence. That says more about the problem spaces and human intelligence that it does about deep learning.
- customer service: chatbot invents refund policy that costs the company money and reputation https://www.wired.com/story/air-canada-chatbot-refund-policy...
- sports writing: a private equity group is running Sports Illustrated - which was already in decline - to the ground
- language translation: not much progress, and more risks of hallucinations
- concept art for films: AI is basically doing plagiarism. And so much AI art is flooding the internet that the AI are basically poisoning themselves in the future. And it still can't get the right number of fingers on hands.
> It is somewhat embarrassing how many tasks, from customer service to sportswriting to language translation to article summarizing to concept art for films, can be handled by deep learning at its current level of competence.
No, it's embarrassing how eagerly leaders will compromise quality to chase automation cost savings. The things you list, especially customer service and sports-writing, cannot "be handled by deep learning at its current level of competence." That doesn't stop people from forcing the results down our throats, though.
Maybe 90% of the results are great, but the other 10% are not only 'not great' but plain wrong and sometimes even dangerous.
And I think gp posts is also showing that the last bump to make it production ready is still not taken and might even be impossible to take.
>“We are still a long way from machines that can genuinely understand human language”. Still true, though some have argued there is some superficial understanding.
>“Elon Musk recently said that the new humanoid robot he was hoping to build, Optimus, would someday be bigger than the vehicle industry”. I expressed skepticism. Still early days, but certainly domestic humanoid robots are not in the near term expected to be a big business for anyone.
>The company’s charismatic CEO Sam Altman wrote a triumphant blog posttrumpeting “Moore’s Law for Everything,” claiming that we were just a few years away from “computers that can think,” “read legal documents,” and (echoing IBM Watson) “give medical advice.”Maybe, but maybe not.” Pending/still true. Two years later we don’t have reliable versions of any of that.
Why does anyone listen to a thing this obvious fraud says?
"The glass isn't full yet. Sure its filling, but its filling and its not full yet, so obviously no matter how much you fill it, we know that doesn't actually fill it."
..is my breakdown on these non-expert non-specific (mathematically, algorithmically, information theoretically, any relevant expertise at all, ...) cynics.
If he had predicted the progress that was made just in the last two years, it would give some crediblity to his opinions about what wasn't going to improve. But he didn't have that insight. He has no idea what's already cooking in the research kitchen.
But if am going to critque predictions, shouldn't I put myself on the line?
So here is my prediction: when general AI decisively advances past us, a lot of people will feel a strong need to write and read articles about how AI doesn't have a soul. And their creations have no heart. And no progress will ever be able to fix this. Despite a lack of concrete or coherent definitions, this will matter a great deal to them.
It's crazy how you can just say things. If by his standards he himself can genuinely understand human language, even given how much of what he says is just smart-sounding hallucinations, then surely Claude 3 well surpasses that threshold.
The secret to grifting is to only highlight your vacuous, unfalsifiable claims. He, like many AI skeptics, is a full-time goalpost mover.
I haven't been really impressed with ChatGPT. It basically seems like a more useful search engine.
And no, AI can't diagnose yet - even expert systems forty years ago did better and it wasn't good enough.
LLMs are trained to summarize text with extremely high accuracy. There is no confabulation of text summarized within the context window. Confabulation only occurs with facts recalled from outside of the context window, which is not a text summarization task. Same goes for human memory. Ironically you have hallucinated a "fact" about LLMs.