Qwen-Robot Suite: A Foundation Model Suite for Physical World Intelligence
qwen.ai
qwen.ai
The TAM for robots is much, much larger than for coding or services, and much more strategic when you think about manufacturing and war-making.
The Qwen "suite" is a workmanlike breakdown with demonstrated tasks that seems to me as an outsider to suggest that one could start building integrated systems this year, and have simple products next year. I'd be very interested in an assessment from engineers from the robotics companies (cars, biomedical robots, manufacturing...).
Elsewhere on HN I see hundreds of comments on SpaceX's long-telegraphed merger with Cursor but no serious evaluation of this.
First, very much expected. Both Google and Qwen have been building explicit spatial reasoning and spatial output capabilities in their models since last fall, gemini 3 was released with support for outputting trajectories for example. I only took a look at Robonav (more relevant for my needs) and its architecture and capabilities are inline with other similar models (eg nVidia's alpamayo).
Second, the overall architecture they describe mirrors what I've been working on: You have general purpose LLM that takes a look at the works and the task in front of it and reasons to break it down into subtasks and tool calls, and you can think of RoboNav and RoboManip as tool calls here. The harness keeps a memory and manages the context of the LLM and tools and keep looping until the objective is complete.
Consider the task of clearing snow off a driveway using this suite: An LLM (Qwen 3.7 plus) takes look at the driveway and decides which areas to clear. The harness then tells robotnav to go to an certain location, then robotnav takes over an runs in a loop until the robot is that that location. Then the harness tells robotmanip to use the plow to clear strip of snow. The harness will then call the planner LLM to plan an execute the next clearing and repeats until the driveway is clear.
So what' the issues? Well, they didn't release the weights, nor the training scripts so you can't actually use it. But also, it's all very research-y still, the models are "small" but still huge/expensive for current edge hardware. You'd still need lots of data collection, HITL, and fine-tuning and evals to make it work for your task. You'd also need a secondary safety system to make sure the models don't wreck something. But overall, I do expect robots to use an agent/model combo like this in prod in a few years.
Well I guess I'll have to have a look!
Why do you hate subscriptions? What if you get a summertime snow storm?
But can it deal with arbitrary lots without extensive premapping, manage piles, handle obstacles intelligently, correct itself (ie spot needs a second clearing ), tackle windrows, etc? It can't, and my hunch is that LLMs are the first tech we have that can plausibly handle all the various cases that a proper robot would need to handle.
Robotics probably will absorb a lot of Rl/diffusion-based tech, with LLM at a high level interface at best.
Having said that, this scheme seems like it might just be a reaction to current hardware limitations. When I saw Talaas demonstrate a 8B model running on a custom chip at 17k Tok/sec, first thing I thought was "wow, you can just run an LLM in a control loop"
how do you figure?
Regard it as market segments. It's not hard to envision eg. agriculture & food processing robotized to the point where no human ever touches your food. A few generations in, and people would see potatoes as "nutrient-containing object that comes from a factory" and forgot how to grow potatoes.
I'm rooting for the 'market segment' where AGI (or ASI) finds solutions to long-standing science questions, that are hard to obtain but easy to verify. Or makes new discoveries. Stuff like cancer research, protein folding, synthetic biology, new materials, battery tech, number theory, particle physics, etc etc.
If I could get an affordable robot to do a subset of them, I'm in the market for one.
I wonder... How can these foundational models actually learn to deal with that without being deployed in those scenarios? It feels like a chicken-and-egg problem: you need a ton of real-world data from chaotic homes to train robust models, but you also need robust models before you can safely deploy robots into those exact homes at scale