- the relatively crude tactile and proprioceptive sensing apparatuses of robots when compared to humans
- the limited availability of multisensory, perception-action coupled training data
Genuinely curious!
Eg, even LeRobot (without proper fingers) can fold clothes now: https://www.youtube.com/watch?v=dPe9v4gqbdg
The labs are spending huge money collecting "multisensory, perception-action coupled training data" (eg, there is the one in NY that gives you free cleaning in return for video data from the cleaner).
Edit: The Gemini Robotics blog post has a video of it tying knots too. That's pretty good.
What are you bearish about precisely?
The trick being to tread continuously through some non-obvious happy path. And average people will be convinced that you really have some breakthrough tech.
But hey, this is not something new. Magicians were taking advantage of such things for centuries ..
> 2. Deploy it somewhere where it won't encounter things it won't handle
Who are doing these? Waymo? how? You're talking BS if you can't elaborate.
After all autonomous vehicles has been well funded research since the 1980s the DARPA grand challenge being one of the previously most important benchmarks.
I think you might just need a history lesson friend
The entire history of this field is precisely that problem and repeatedly demonstrated
I don't think I was clear and explicit, it being tedious to write, and I apologize for that. I also apologize for shifting the goalposts as I had not written out my own position, which is not exactly in the "grandparent commenter"'s position (that I had not previously given enough attention understanding), but it is also not in agreement with yours. I don't mean to say that we are not presently in the "bitter lesson" (your idea of what the bitter lesson says) regime. I definitely think that a lot of progress can be done right now by emphasizing the humanoid robotics platform as a foundation. What I mean to say is that I don't know if that platform with the hardware we have today is sufficient for parity with human housekeeping tasks in the domains that we wish it to have parity. The bitter lesson itself (not your understanding of it) is in fact silent on this as it is in relation to feature engineering, where it is a clear point, but you seem to be adapting it uncritically wholesale to mean something more than what it is written about. My position is that, it is unclear whether today's sensor platform is sufficient for parity. It is less strong than the blog post author's "Why Today’s Humanoids Won’t Learn", it is a "We can't say whether or not today's humanoids will learn", but it is something that also contradicts a "the bitter lesson means today's humanoids will learn" thesis.
The self-driving car supports my claim, because after so much investment in capital and time, we ended up with a car with comparatively expensive LIDAR sensors as our preferred platform.
Plenty of hecklers were saying "you can't self-drive on cameras", and some still try. But Tesla's self-driving on cameras, and it seems to work fine. While Waymo's self-driving on fat sensor stacks, and it also seems to work fine. Sensors don't seem to be a differentiator of self-driving performance.
I don't think anything about self-driving tech supports your claim. Tesla was bullish on AI all the way, and Waymo has also shifted towards highly integrated end to end AI. It's the AI advances that make self-driving tractable - not anything else.
I can't evaluate how true or sensationalist this story is, but this bearish article suggests to me that Tesla robotaxis today isn't yet the success you are painting https://electrek.co/2026/07/03/tesla-robotaxi-miami-service-...
Has it been demonstrated? Or is it just that such things now get a lot of funding now, for no good reason?
All three offerings from the linked blog post are either Vision LLMs or Vision/Action LLMs.
That does not help a lot.
But your entire premise is wrong regardless of that.
Even if VLAs were forever bound to outputting text, you'd have to prove that they're fundamentally incapable of emitting text that maps to useful action sequences. No proof of that whatsoever - and plenty of empirical evidence suggests otherwise. Even non-specialist LLMs like ChatGPT are getting better at controlling robots and navigating 3D environments, if slowly.
You don't understand what I am saying. The crux of your misunderstanding is here
>emitting text that maps to useful action sequences
If you have a static mapping from text to action, then you are throwing away all the advantage of using an AI. The whole point of AI is that you can get an output from an input without explicit mapping. So If you use explicit mapping anywhere in the chain, then you lose most of the advantage of using the AI.
So if your hardware, physical vocabulary is limited, like move left/right/up/down then what you say could work. But something that have the dexterity of a human form, this vocabulary is nearly infinite. You won't be able to use explicit mapping there.
Your entire premise is wrong.
Modern action decoders are different, and usually take the form of neural networks trained end to end jointly with the rest of the model. Not fundamentally more expressive, just more in line with what we want.
So What is LLM is used here for? It is used for mere translation between different robots. So it is mostly symbolic translation.
What I am talking about is to translation LLM inference directly to movements. For example, if you ask an LLM, how do I open the microwave door? It will list the steps. I am talking about a system that can go from "put the thing in the microwave", to action steps, without having to never once demonstrate it physically, and do it just from LLM inference.
In short, the way LLMs used here is not (categorically) the way I was asking about.
https://arxiv.org/pdf/2505.23705
https://www.pi.website/download/pistar06.pdf
https://www.pi.website/download/pi07.pdf
The thing literally has a diffusion "action expert" sit in the same attention system as a pre-trained VLM. And the VLM itself is ALSO trained to generate raw actions as a part of the training recipe (the first paper) - it just doesn't do it at inference time. What the "action expert" does is parallelize the action generation process - based on VLM's internal states.
It's exactly the thing you claimed to be impossible. Described in detail in a paper from 2025. What's your excuse?
You already downgraded your claims from "LLMs are irrelevant to robotics" to a measly "you can't train a useful robotics LLM because there's not enough data". And you say that while looking at an LLM that was pre-trained on all of internet scraped and only then reused for robotics.
Both the pool of robotics-relevant data and the performance of foundation model LLMs grow over time. All the companies that are serious about robotics are serious about scaling up data collection.
I'm not going to claim that this "LLM core" approach is the best approach to AI robotics possible - but if you're betting on it failing outright, you're going to be fighting uphill.
This was the claim from the very beginning. You should have asked why I think what I think, instead of leading with "the entire premise is wrong!"...
Your entire premise was wrong at every point, and now you're trying to wriggle your way out of admitting it.
Prove it!
Read the command line prompt: --task="pick up the red cube"
So it should be something like, "put back this slipped cycle chain back on sprocket"..
Read the command line prompt: --task="pick up the red cube"
There is a gif directly below it.
This is a completely open source model and arm you can replicate yourself.
This isn't true.
Obviously there is a lot of variety in architecture, but in the prototypical example there are vision and languages encoders and an action decoder which decodes direction into action steps. Eg, Hugging Face SmolVLA:
> Specifically, the VLM processes sensorimotor states, including images from multiple RGB cameras, and a language instruction describing the task. In turn, the VLM outputs features directly fed to the action expert, which outputs the final 3 continuous actions.[1]
Or NVidia's GR00T N1:
> A diffusion transformer (DiT) processes the robot’s proprioceptive state and action, which are then cross-attended with image and text tokens from the Eagle-2 VLM backbone to output the denoised motor actions.[2]
(Emphasis mine)
The Bitter Lesson Rich Sutton March 13, 2019
http://www.incompleteideas.net/IncIdeas/BitterLesson.html
And even fewer are aware of the author's follow up on what his article says about the current trend in AI:
Silicon Valley Doesn't Understand The Bitter Lesson – Richard Sutton
Modern robotics is, at its core, not a hardware problem. It's an AI problem. We have plenty of headroom in the hardware - what we don't have is an AI good enough to utilize it. We don't know the practical limits of current hardware because we can't make a robot AI that would make the hardware a meaningful bottleneck.
Today's robots don't fail at tasks because they have poor fingers. They fail because they don't know how to perform those tasks. If you put an effort into solving that? You get demos like: Gemini Robotics 2 tying a garbage bag. Take one long look at that and think of manual dexterity.
Human body is crude and suboptimal in a thousands different ways, and all of it is salvaged by advanced intelligence.
As usual with robot tech demos: WYSIWYG.
Every time you see something that "suggests a very specific, very precise, "algorithm" taught in an imitation learning session"? Scale the imitation learning up x10, x100, x1000, and it suddenly generalizes!
I'll be honest: I don't see what you see. I don't see anything that would suggest this algorithm is so brittle there's zero transfer to "even other garbage bag strings". AI robotics isn't innately brittle like conventional robotics is. But even if you are, somehow, completely right on that? Teach a hundred "very specific algorithms" like this - and watch them fuse into a manifold of algorithms that can be applied to different problems as needed.
And that is what you need. If an algorithm for "tie a garbage bag with current generation robot hands" exists and can be learned by an AI, then the gains from getting better AI are far from exhausted. The limits of robotics are the limits of AI.
This is why every AI robotics company is saying "we need more data". They understand what they're dealing with. They looked at the scaling laws and went "robotics isn't magic, that curve applies to us too". I don't get what makes people see robotics as a special magic thing, that makes them look at the advances in robot AI and say "this is intractable" and not "this is hard". It's hard. We're getting through it though.
Before I put in the effort to reply in good faith I have to know: do you think we're going to have a conversation or are you going to fulminate and scold me like some kind of all-important authority (which I have to say you clearly are not)?
To clarify, I'm happy to have a curious and respectful exchange.