RoboAgent: A universal agent with 12 Skills | Hacker News Reader

RoboAgent: A universal agent with 12 Skills

robopen.github.io

145 pointsreqo19 comments

Open articleView on HN

The 12 skills are

- Flap open/close (oven door)

- Slide open/close (drawer)

- Slide in/out (slideable shelf)

- Cap open/close (screw caps)

- Pick/place (for relatively easy situations on flat surfaces, not bin picking)

- Wipe

"Wipe" doesn't have a reverse operation, and #12 is not illustrated.

The main thing here is that the robot has much force sensing, and interpreting that data is hard. So they had people do this with a teleoperator with force feedback, and fed that into a machine learning system. What this is really about is how to use force feedback data effectively. People have struggled with this for key-in-lock and part assembly for decades.

Seems like good work on a longstanding hard problem, but not a breakthrough.

> "Wipe" doesn't have a reverse operation

Wax on/Wax off obviously.

Still no nunchuck skills, bowhunting skills, computer hacking skills...

I'm sure the team doesn't want to add competition to the dating pool.

Related: RT-2 ([1]). The RoboAgent compares itself with RT-1, but not with RT-2, which is a big improvement upon RT-1.

Disclaimer: I work in the same group that made RT-1 and RT-2, but not a direct contributor to that work.

1. https://www.deepmind.com/blog/rt-2-new-model-translates-visi...

Somewhat jarring but really good (contradictory, I know) presentation format. A few tweaks and it would be perfect. Very nice project too.

Well done, dudes! If I wanted to get my hands on a robot that could be used with this, where should I look? Exciting times.

"What is my purpose?"

"You pass butter."

"...oh my god."

"Welcome to the club, pal"

I wish I had 12 skills

but then it’s over for you the moment there’s someone with 13 skills

Who of course, would be immediately obsoleted by the Chinese, who skip the unlucky number 13 and therefore have 14 skills.

If by skills, we mean this is something we rely on as a profession, I'm happy to let the robot take these jobs. As long as I can handle tapping, scrolling, and clicking, I'm good to go.

Edit: Or I can get better at Vim controls and toss out scrolling and clicking.

13 is only unlucky for Christians, because of the Last Supper.

14 is unlucky for Chinese, because it sounds like 'is dead'.

A talented Chinese Christian might skip both and go directly to 15 skills.

Examples given are so underwhelming it actually kills my enthusiasm for robotics.

Post comparible results to Boston Dynamics or you’re old news.

Where can I find Boston Dynamics' GitHub for open source resources I can use in my own products?

This is a negative comment that adds little to the conversation and belittles hard work being done. Comments like this is why many people are afraid to share their efforts - I ask that you consider people on the other side of the screen.

Also, Boston Dynamics largely uses classical non deep learning techniques. We're still exploring AI and deep learning techniques to hopefully create such performance.

Quite the contrary, I think such comments are sorely needed in AI and its intersection with robotics. If we have a problem in Deep Learning today it’s that we’re over sharing. There are a ridiculous number of papers, and none of them add any value whatsoever to the literature. We’ve gone past the stage where people are sharing small incremental results and clogging the space, now they’re actively covering up their lack of new results with nice presentation, acronyms, buzzwords and making a paper out of it. We really need to be discouraging every run of the mill result from being published as a paper, and holding high standards for them (like say Boston Dynamics Tier) doesn’t seem like a bad idea!

This work (RoboAgent) is not a run of the mill result. It might look not as shiny as Boston Dynamic does the demos, but it's an actual improvement of what's possible.

One of the biggest issues with Robotics is that it's really hard to program the robots. The ability to set the tasks to a robot with a language brings the complexity down and makes it potentially accessible to regular people. We are not there yet, but RoboAgent is a fine step towards that direction.

Let's hope that the code will be released soon as promised at https://github.com/robopen/roboagent/. RoboAgent is much better than RT-1, and comparable to RT-2. But the latter is not open-source, and so has a limited usefulness in advancing the state of art (unfortunately).

Dismissing great results only because they don't have a video with $100k production budget is not what science is about.

Can a Boston Dynamics robot do these tasks if you just ask it?

I don't think Boston Dynamics robots can do any of this on their own. The key here is that these skills are learned in a general manner. Nothing like the pre-programmed hard-coded choreography demonstrated by Boston Dynamics (which is still extremely impressive).

Reply on news.ycombinator.com