Show HN: Robot Utility Models: open-source robots working in new envs zero shot
robotutilitymodels.com
Why is this exciting?
The state of the art for robot learning thus far has been reliant on pre-training/fine-tuning paradigm: models are first trained on large datasets, then you bring it to your own environment, collect more data, and fine-tune the model on that data. This is not ideal if you want to make robots that work in every home out of the box of course.
We're finally free of this status quo! We showed experimentally how the key to this discovery is getting data that is both HIGH QUALITY and DIVERSE. High quality means don't just let anyone collect robot data, and diversity means go to as many different environments as you can to collect your data.
We have a final piece that helps the robot get even better, which is simply asking GPT! You give a series of your environment images to GPT and ask it if the robot failed or succeeded. If it says the robot failed, you reset and retry! Just this simple trick helps the algorithm a lot.
We've also open sourced all of our code, models, data, and training code. We believe people should be able to go out there and create their own utility models for tasks that they care about, which I find very exciting!
Would love to hear your thoughts, or suggestions for next steps.