Using OpenAI Gym to train an open-source 3D printed robot | Hacker News Reader

Using OpenAI Gym to train an open-source 3D printed robot

github.com

135 pointsnicrusso726 comments

It's unfortunate that we still don't have cheap torque controlled robot actuators. Controlling force rather than position led to a drastic improvement in walking robots. I would also highly recommend using something beefier than an arduino to control a walking robot as most approaches to walking robots today rely on performing fast optimization. MIT's minicheetah has used a control approach that involves simulating the robot about 0.5 seconds into the future 50 times a second.

Why does torque controlled work better?

In interaction with anything rigid (or even deformable), position control does not mean anything. Force (or torque) control defines the actual physical interaction.

We actually have very cheap and pretty powerful position-controlled actuators (hobby servo motors). Attach any kind of spring and displacement measurement device (potentiometer, hall sensor, optical, LVDT, etc) and voila, instant torque controlled actuator.

You can look up Series Elastic Actuators for more info or use this article as guidance (any spring will do as long as the force range and spring constant is adequate).

https://www.sciencedirect.com/science/article/pii/S240589631...

The servo they use is $493[0], at that price I wouldn't necessarily consider it a hobby servo. The control for that specific series elastic servo needs some work. There have been other attempts at making cheap series elastic actuators. An interesting one was the programmable spring work[2][3]. Although one problem with series elastic actuators is that they can be difficult to control because of the compliance. Force servos were also an interesting attempt at doing cheap force control[4] and by using load cells they avoided the compliance problem. Unfortunately, force servos did not have any position control.

[0]http://www.robotis.us/dynamixel-mx-106t/ [1]https://www.youtube.com/watch?v=sjFR4ACVLmk [2]http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.147.... [3]https://www.youtube.com/watch?v=_g79mOSvSsE [4]https://www.youtube.com/watch?v=sjFR4ACVLmk

It is also possible to use current sensing to measure torque.

ODrive + brushless + encoder (~150€ per axis) can provide current measurements, though you may need to create some PID loop.

If the brushless solution is still too expensive. You can use 775 brushed motors + encoder + BTS 7960 driver + (optional gearbox) (They have current sensing by using additional resistance (Edit:and RC filter) ) (10€-50€ per axis from china). Although that's not powerful enough for jumping robots.

I do agree, this is one of the many improvementents/experimentation the project needs. Actually, the Minitaur example come with a torque controller implementation (https://github.com/bulletphysics/bullet3/blob/master/example...), needs tests. btw I'm going to use a Jetson mini as controller!! :)

I recently came across this life sized robot you can 3d print: http://inmoov.fr/build-yours/

I’d love to try it. I couldn’t determine if it can walk or not.

Looks like the legs are just for show [1], but I bet you could kitbash something

[1]http://inmoov.fr/legs-non-motorized/

I've been wanting to build this for awhile, it looks pretty awesome.

Wow. Reach out if you do. I want to do it too but it looks like such a huge undertaking.

Is the aim of the project to ultimately make the printed robot to run and walk?

Maybe the next step will involve fitting servo motors on to the robot?

IIUC, SpotMicro seems to be a project that already has this part sorted out: https://youtube.com/results?search_query=spotmicro

I don't see any of those videos actually demonstrating Spot Mini walking successfully. Some comments suggest that the motors are not strong enough, or the body is too heavy.

I've already printed and assembled one, the creator posted 2 videos in his thingiverse repo https://www.thingiverse.com/thing:3445283

Is there any research on training a robot to walk from scratch. Ie having it learn from the ground up just trying different motions.

I'm not sure about doing that with physical robots, but I've seen a couple interesting iterations using simulated robots

https://www.youtube.com/watch?v=gn4nRCC9TwQ

Well the very first attempt this guys made (https://arxiv.org/pdf/1804.10332.pdf#subsection.6.1) was let the model learn from scratch (open loop, large bounds feedback). An agile galloping gait emerged automatically also in my simulation (https://github.com/nicrusso7/rex-gym#galloping-gait---from-s...), even if the gait was 'noisy'

As per the tradition [0]:

In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6.

“What are you doing?”, asked Minsky.

“I am training a randomly wired neural net to play Tic-Tac-Toe” Sussman replied.

“Why is the net wired randomly?”, asked Minsky.

“I do not want it to have any preconceptions of how to play”, Sussman said.

Minsky then shut his eyes.

“Why do you close your eyes?”, Sussman asked his teacher.

“So that the room will be empty.”

At that moment, Sussman was enlightened.

[0] http://catb.org/jargon/html/koans.html

The more times I read this, the deeper it gets, and the more I appreciate modern machine learning.

I've seen a crawling robot trained with a string which would pull it back to the starting position after each episode so it could automatically try over and over again. I'd love to see someone doing that with a biped and just lift it up on a hoist to reset its orientation and position whenever it fell over. But really, it might not make a lot of economic sense if you can simulate the physics like these guys are doing.

This is awesome! I wish there was a video showing the results. This might be my next project!

Thanks! I'm currently working on the 'knowledge transfer', the aim is to build an almost-real-time controller for the robot - I'll probably start with a web app. Any contribution is welcomed!! :)

Yes!!! Is it part of the same repo? I'm a MechE but my software is decent enough for prototypes!

nope, I'll publish rex-robot repo asap ;)

even cooler if the anatomy of the robot were optimized in the gym as well, and then you just ship it to the printer

Reply on news.ycombinator.com