Gemini 4 Argon for Robotics
developers.googleblog.com
Main thread: Gemini 4 Argon - https://news.ycombinator.com/item?id=50073879
developers.googleblog.com
New trucks optioned out cost $80k+ and plenty of people pay for that without having much justification of any kind for it.
In the US, certain purchase categories make taking on debt to cover upfront costs trivially easy. Open robotics platforms are not one of those categories, while automobiles are.
Meanwhile a guy who made a fortune in tech optimizing ad placements (not saying that's OP) can pick one up on a lark, and as you said it usually ends up in the garage next to other expensive toys like an $80k truck or autonomous Tesla. Also the $80k trucks are a great example because they're more of a status symbol, just like the robots will be.
It's 1.23 and weights ~30kg. Even though only the R1 Edu can be in theory programmed, it can be easily modded to enable SSH and root access to deploy custom movement policies and interact with the different on-board daemons to control the movements, grab the video data, etc.
I bought a second-hand Intel DeepSense camera and plugged it on one of the spare USB ports available in the main board to have 3D sensing (even though I really need to play a bit more with it). Right now I am playing with MuJoCo for RL training of new policies, integration with VLAs like NVIDIA Isaac GR00T and integration with LLMs.
I have to say its being a wild ride, and despite the small size of the robot, I have already injured myself with it when playing with the walking policies (If you activate a policy while its hanging, it starts literally kicking and punching you...) I cannot image those owning with the bigger brother of it, the G1...
Based on my very limited knowledge... if you would have dexterous hands with sensitivity on them you could theoretically train a VLA model to do that task, planned by a high-level model like Argon 4, that orchestrates everything. That would as well require in first place a VLA that has similar tasks in its training set and fine-tune it by means of recording hundreds of examples performing teleoperation of the robot hand and arms movements.
Not easy, but definitely getting more and more doable these days.
PS: Didn't spend 20k on hands for my 6k robot, but can confirm it has the cable harness to support them :D
What's the latency like? Is the llm used to create an instruction set for the robot? How is it interrupted for reflex actions like controlling stumbling?
In any case latency is acceptable for navigation I guess, but avoiding moving obstacles is definitely a no-go right now.
What happens when people decide to hack the Unitree robots and have them pick up knives and stab their owner and then go on a spree in the street?
Before people say: "But that's never going to happen, nobody would ever do that!" I'd like to state again the fact that people did just remotely hack fridges to ruin food.
"But ruining food for everybody is nothing, it's just kids playing around, people would never hack robots to hurt other humans!".
Is that what people really think?
Or do they think that "this time programmers managed to create secure software that cannot be hacked into"?
The difference between you and most professionals is that we don't play. Groceries weren't playing with their refrigerators, they were attacked, the government can find and punish or kill the criminals if they continue their pattern of failure to bring justice the world will move past them.
- Zero shot control using a visual harness and action primitives. I've seen this from a few other places as well (https://tml.stanford.edu/homebody/) and it's essentially having a generalist LLM command a robot via a set of predefined actions. Many robots and many tasks can probably be done with this approach, since not everything needs super intricate 6DoF arms or dexterity
- simulation bring-ups: I have a pipeline that takes in satellite imagery and reconstructs the lots and houses in a 3D environment and snows on it so that my virtual robot can learn to clean. With these latest models, you can probably create a higher fidelity digital twin more directly.
- self-improvement: These models (Argon, Astra, etc) can't operate the robot itself, but would be able to create data to distill into smaller models you can actually utilize.
I'd much rather see videos of a prototype than a fictional video that has no bearing on how the thing actually works. Especially since you list pricing on your website, making visitors think it's a currently-operating business.
Seeing prices like 5k EUR is a bit too much for a hobby.
Aren't we still like a decade away from an Argon model being able to run on battery power for any reasonable amount of time that would fit on a robot that isn't the size of a school bus?
Add on another couple of years for the speed to get to what it likely needs to be to be generally useful instead of narrowly useful?
I mean - I guess we're maybe 5-7 years away from a reasonably priced robot you could plug into the wall like an old vacuum cleaner (that runs on this type of tech).
Also, your model probably doesn't need like a lot of its knowledge to control a robot.
From a cost perspective, I just can't imagine that heavily, heavily specialized models wouldn't crush a general model, though - which Amazon already does.
Robotics tends to be at the worst possible intersection point of all three of those: limited power budget, relatively small model size and cheap enough to be affordable when embedded into an appliance.
It just seems like jumping the shark a little bit.
It's basically completely unviable for a few years still.
I don't think it's viable in the sense that they are still in a demo stage.
Also, I would assume these models are going to get bigger before they get smaller (but they will get smaller over the long run)
For a set of tasks that almost no one would use a robot for...
My assumption is the hardware will have one/several smaller models running on it for locomotion, balance, etc.
Then the hardware "phones home" for decision making and planning.
Essentially the planning doesn't need to be close to the device as long as the local models can handle some part of the plan execution and make sure the robot doesn't fall over. The latency shouldn't be an issue. In fact more powerful centralized compute quickly creates an advantage over slower local compute in terms of end to end planning.
However, if the robot is in a contained environment like a home, there is less third party liability than for something like vehicles. Not none, of course, but less. And when you can get your arms around third party liability, first party liability can be better controlled by contractual agreements.
In general I expect on-device motor execution will allow it to safely come to a stop or enter a safe position if it loses the link to the execution planning cloud model.
Sooner or later, the models will stop improving. After that happens, the best models will be etched into model-specific ASICs with weights fixed in mask ROM (like Taalas is already doing), which will result in massive, at least order of magnitude inference efficiency increase.
And although the industry hit a wall in density/area, they switched to scaling in 3D or glueing chiplets together. Moore's Law is still working if you change the definition to transistors/integrated circuit.
For data bound things, there’s also fundamental laws of information theory, far stronger than physics theories, since they’re theorems about math. And LLMs can only compress knowledge so far, and we’re about at it now.
Robots stand-alone AI do not really benefit from things like batch mode so you want to keep your LLM as high level as possible and your robot's subsystems as autonomous as possible, very light coupling between the two, definitely nothing time-critical. And that's the challenge: in robotics almost everything is time critical.
I think someone should make a game based on this concept.