Hacker News Reader

Comment by randomNumber7 | Hacker News Reader

What ML algorithms do you intend for full autonomy? Multi Modal LLMs for planning that control the robot by generating s.th. like code? Or s.th. that requires more learning from the environment?

When I click "get in touch" on your github I just land on the website where I can buy the robot. Building the hardware for an autonomous robot is orders of magnitudes easier than the control. Do you think anyone with the capability do develop an autonomus robot will buy this and then just give you the code because its open source?

View on HN

My overall plan is basic joystick control -> VLA with RL -> self-supervised embodied representation -> end-to-end RL -> end-to-end control. I suspect there will be some very good multi modal models coming out in the next few years which we might use as base models, although more likely, we will adapt their techniques to work on data from our own robot.

I agree that the hardware is easier than the software - I am a software guy, personally, but I felt that it was important to do the hardware first so at least we can have a baseline product which we can offer to people. I would personally like to work on this software problem (or rather, build a company to work on this problem), and this seems like the right way to go about funding working on this problem.

I like the K-Sim Gym. Im looking forward to fiddle with it a bit when I have more time. I could see that you get something usefull out of people competing on your leaderboard xD

It's my hot take that the next big ML breakthrough needs s.th. that learns from its own actions in an environment, so this goes in the right direction imo.

On the other hand a lot of big companies struggle with self driving cars even though they predicted to build this years ago. Also probably all big AI companies work on AI for autonomous robots. Where do you intend to do s.th. different to get a shot at competing with them (when they have so much more capital)?

I really do think that building through the open source community is the best way to compete with the big players, even without having a lot of capital. Of course, it doesn't mean we can't execute well, but I do think it's a good way to make a lot of progress without spending a lot of money.

Reply on news.ycombinator.com