Isn't this a real big deal? They manage to train a foundation model that connects the physical understanding gained from unsupervised training on images and videos with the real physical understanding necessary to control a robot. They have impressive videos to show for it. The approach seems indeed scalable and generalist. I've thought for a while that the keystone missing for household androids is connecting the understanding of large multimodal language models with physical hardware. To me this looks like exactly that. It actually makes me optimistic that we will see household robots within a decade. Now I wonder why Tesla, Figure etc. are messing around so much with Teleoperation if this indeed works. Maybe I don't understand what's going on.
I think the fine tuned policies are still very brittle, but I agree that this is super promising. It's also one of the most open (the model is still closed) research blogposts we've seen from any private embodied AI lab
If nothing else, the uncut video of the robot autonomously folding laundry for 10 minutes is incredibly impressive. Regardless of current robustness, or how long it took to get that specific clip, or how engineered the prompts are, this is no small feat.
Physical intelligence's first model: π0. In the blogpost and included paper, the company's robot foundation model debut features a mobile bimanual robot autonomously folding clothes, bussing dishes, putting eggs in a carton, and folding a cardboard box, supervised by stern researchers. They go into some depth of their training recipe, which includes a pre-training recipe with OpenX and self-collected expert demonstrations on 68 tasks on 7 different robot embodiments. SOTA results on several internal task benchmarks on real world tasks are shown, although much of the benchmarks are currently limited to 10 trials each.