My overall plan is basic joystick control -> VLA with RL -> self-supervised embodied representation -> end-to-end RL -> end-to-end control. I suspect there will be some very good multi modal models coming out in the next few years which we might use as base models, although more likely, we will adapt their techniques to work on data from our own robot.

I agree that the hardware is easier than the software - I am a software guy, personally, but I felt that it was important to do the hardware first so at least we can have a baseline product which we can offer to people. I would personally like to work on this software problem (or rather, build a company to work on this problem), and this seems like the right way to go about funding working on this problem.