Ask HN: What are the modern approaches to robotic control? | Hacker News Reader

Ask HN: What are the modern approaches to robotic control?

43 pointsbistable20 comments

I'm eager to find references on modern approaches to robotic control.

The coordinated robots from Boston Dynamics (acquired by Google) impress the most, with their ability to react quickly to perturbation and move competently over difficult terrain.

I assume they are not solving inverse kinematics problems continuously, and also assume that much of their progress was made before the deep learning revolution, so maybe they don't even use neural networks either.

So are they using any reinforcement learning? I have no idea so would appreciate any quick summaries or pointers to relevant information.

What other companies are working in robotics at the same level as Boston Dynamics?

Approaches to robot perception, path planning (e.g. A*), and environmental mapping (e.g. SLAM) are also very interesting, but not the topics I'm interested in for this question.

View on HN

You say you're not interested in path planning, but path planning and control are really on the same spectrum, which is motion planning. For instance, where to next put big dog's leg is a control problem, but also a path planning problem (how do I get the leg there? Where is the best place to put the leg? Does the leg collide with anything?). If you can't answer these questions and are blindly putting your leg wherever, you're not in control of anything.

Here's a good paper on RL in robotics: http://www.ias.tu-darmstadt.de/uploads/Publications/Kober_IJ...

RL is not widely used for control, but it has yielded some impressive results. In my experience I had a highly dynamic system for which I built a hand-tuned model for motion planning. I also built a RL model and trained it using the hand-tuned model. The RL model performed more than 50% better than my very best efforts.

Also, some of my favorite textbooks:

Principles of Robot Motion by Choset et. al.

Statistical Robotics by Fox, Burgard, and Thrun

Linear Systems Theory by Hespanha

Not my field, but in case this link is helpful:

"Planning Algorithms" by Steven M. LaValle

http://planning.cs.uiuc.edu/

>This book presents a unified treatment of many different kinds of planning algorithms. The subject lies at the crossroads between robotics, control theory, artificial intelligence, algorithms, and computer graphics. The particular subjects covered include motion planning, discrete planning, planning under uncertainty, sensor-based planning, visibility, decision-theoretic planning, game theory, information spaces, reinforcement learning, nonlinear systems, trajectory planning, nonholonomic planning, and kinodynamic planning.

For more generalized information on how you bring a machine from state A to state B, you may want to read up on Control Systems [0], one of which is the PID controller [1]. It's used in everything from industrial water heaters to quadcopters. It wouldn't surprise me to see them also being used at some level with more advanced robotics.

[0] https://en.wikipedia.org/wiki/Control_system [1] https://en.wikipedia.org/wiki/PID_controller

One step further for linear motion control systems like CNCs and 3D Printers is a project called TinyG which effectively democratises the application of 6th-order jerk-controlled motion planning in real-time. https://github.com/synthetos/TinyG/wiki

Very natural looking movement from these systems! https://youtu.be/aucE49ZBXx0?t=46

PID is a generalized control algorithm that performs poorly unless your system is entirely linear. It is overwhelmingly used in industry, however, because it is simple, and anyone can drop in a PID block and adjust three constants to get semi-OK response without understanding anything about the dynamics of the system they are controlling.

For serious control systems you can look up state feedback control, model predictive control, and nonlinear systems control (to name a few very broad categories out of many possible options).

Yes, and I'm compelled to add that robots are typically actuated by servo motors and/or hydraulics, which are very commonly under the control of PID loops. Those PID loops allow the actuators to reach their set-points (whether position, velocity, etc.). To obtain 'smooth' motion, those set-points are usually streamed to the actuator at 60+ Hz, and it is the generation of those set-points that can become very complex; the stream of set-points is not commonly generated by the output of a PID controller.

PID control is rarely used for professional servo applications, with the exception being amateur and lower performance systems (radio controlled, etc). This is especially true with PM servo motors because these machines are inherently nonlinear and cross-coupled. PID doesn't deal with either of these issues (nonlinearity or cross coupling) and therefore offers extremely limited performance.

The industry standard update rate for servo systems is ~1kHz (although it depends on the application), and I have seen systems with >5kHz torque bandwidths. The torque dynamics associated with a typical PM machine used for servo systems are easily in the 10's or 100's of microseconds, so 60Hz control would not cut it.

Again, yes, but to clarify, the 60+ Hz I was referring to was not the servo loop update rate, but the rate at which (loosely speaking) the servo loop setpoint is updated. (1)

For example, KUKA robot arms can operate in a mode where a motion path is planned and a sensor on a tool tip can make slight adjustments to the motion path on the fly. The points on those motion paths (as well as corrections) are updated every 4 or 12ms (83.3 to 250 Hz).

My point is that servo motor drives do indeed implement (sophisticated) PID controllers for current, position, velocity control loops. (2)

Obligatory Defensive Writing:

(1) Typical industrial servo motors implement sophisticated control over the velocity and acceleration profiles of the point to point moves. Anyone interested can look up the DS402 standard and take a look at the motion profile modes of operation.

(2) The PID loops implemented in servo drives are way more sophisticated than the canonical PID control loop equation (https://en.wikipedia.org/wiki/PID_controller). Nonetheless anyone who finds themselves manually tuning the servo loops for an industrial servo motor will surely find themselves setting proportional and integral gains.

Source: Commissioning servo motor drives is a part of my job.

Just because controllers have proportional and integral terms does not mean they are PID controllers!

I'm not just being pedantic here, there is a world of difference between a modern servo controller and a simple (or advanced) PID controller. Good servo controllers include completely different topologies including state feedback decoupling, disturbance decoupling, state tracking feedforward terms, advanced notch filters, sliding mode gains, and many other techniques. This is not just an advanced PID loop, but a different controller design method altogether.

Source: I design servo (and other motor) controllers for a living

There is a surge of interest in learning control policies end-to-end. Many of Sergey Levine's recent papers are relevant: http://homes.cs.washington.edu/~svlevine/ (there are also some talks linked there).

It's been a few years since studying in this area, but one approach we used to use for modelling and control of manipulators (mostly industrial robotic arms) was Screw Theory.

Here's a recent paper using one of these approaches: http://www.intechopen.com/books/international_journal_of_adv...

One approach that could be used for a system like Big Dog would be a feed-forward control loop with kinematic/dynamic modelling of the robot. These approaches use knowledge of the system dynamics to predict the output based on changing inputs or disturbances.

You'll really be missing out if you don't look into the work of Emo Todorov:

http://homes.cs.washington.edu/~todorov/

He's done some amazing work recently in helping solve problems surrounding movement planning (not really the same as path planning like A*), so you can tell a robot to do something general like "stand here", and it treats the movement planning as an optimization problem and the dynamics of its 'body' and physics as constraints. The resulting behavior is eerily lifelike.

they use something called "force control" which I think uses a spring to act as some kind of tendon.

http://news.nationalgeographic.com/news/2003/03/0311_030313_...

http://web.mit.edu/spotlight/archives/troody.html

Mentions some of the engineers at Boston Dymanics including the inventor of troody. https://prezi.com/tn5kji7ehk3u/prezi-project/

more info here:

http://news.mit.edu/2001/dinosaur

We use series elastic actuators, or springs connected to motors, and low-stiffness control, so the joints are looser, and more biological."

HOW IT WORKS Troody, which weighs about ten pounds, has 16 joints and 36 sensors. "Every joint has a position and force sensor," Mr. Dilworth said. The robot also has a vestibular system -- the equivalent of an inner ear -- that it uses for balancing, and an onboard computer that automatically runs a walking control algorithm.

some more info:

http://sandbox.softroboticstoolkit.com/book/series-elastic-l...

We use series elastic actuators not so the joints are more biological, but mainly so that motors don't get creamed when the robot bumps into something and so that the robot doesn't cream whatever it bumps into. The second reason why we do this is to get force/torque control. Series elastic actuators measure force or torque by measuring the deformation of a spring. You get increased force bandwidth at the expense of decreased position bandwidth

If you're interested in walking, the Florida Institute for Human and Machine Cognition (ihmc: http://robots.ihmc.us/) came in second place at the DARPA Robotics Challenge using force control and capture point dynamics. They do primarily software, but their walking is quite stable.

Sample efficient methods using Gaussian Processes are interesting.

http://mlg.eng.cam.ac.uk/pilco/

Google "underactuated"

And see the MIT "Underactuated Robotics" book (Free online) [1] and associated MITx course [2].

[1]: http://underactuated.csail.mit.edu/underactuated.html

[2]: https://www.edx.org/course/underactuated-robotics-mitx-6-832...

Reply on news.ycombinator.com