Hacker News Reader

Comment by 6gvONxR4sf7o | Hacker News Reader

This looks neat, but anyone interested should know there's also a whole mature field to solve similar problems. The field of control theory has some really simple and really robust methods for getting a system from state A to state B. If you aren't trying to learn how the system works simultaneous to your algorithm being deployed, it can be a really a good fit.

This book and control theory solve different problems. I'm just commenting because I sometimes see modern ML folks trying to apply super duper general techniques to problems that are easily solved specifically with simple undergrad-level techniques from the specific fields (including this material to control problems). Also, I just think control theory is really cool and perhaps under-discussed in "cool math/eng stuff" circles.

View on HN

You are right that sometimes people use much more sophisticated algorithms than are really required to solve a problem. My experience is that the simplest approach is often the best one in the long term. There are actually deep connections between problems in control theory and the topics in this book (e.g., LQR control), though these different communities often use different notation. The focus of the book is more on higher-level decision making, but the execution of the decisions---such as motor commands to an actuator---are often best done through PID or some other method that can be found in a control theory textbook.

Control theory is indeed, quite cool, and under-discussed in most circles.

Also, unfortunately, classical control theory is primarily good for linear time-invariant dynamics in the frequency space of the Laplace transform. If you can't locally linearize your model and/or need to learn a model, classical control approaches are underdeveloped, and everyone has switched to optimal control and RL.

I think you might mean something different by optimal control than I would. I'd bucket that into control theory, but it seems you're drawing a distinction.

I would distinguish between classical control and optimal control. In the former, you don't have a (Hamilton-Jacobi-)Bellman equation to solve, while in the latter, the approximate dynamic programming is The Hard Part.

This is neat - specifically because what you point out is actually true. The primary author is a professor and the secondary author is a Phd graduate - both in the Stanford Aeronautics and Astronautic Department [1][2] - in the Dynamics and Control Group.

A brief skim through of the book shows many references to Control Theory texts.

[1] https://mykel.kochenderfer.com

[2] http://timallanwheeler.com/aboutme/cv/Wheeler_CV_201804.pdf

That explains why there is not as much traditional Decision Theory in the book as one would expect from the title, such as Peter C. Fishburn's work, issues in MAUT and MCDM, seminal work by Bouyssou, Vincke, Pirlot, Keeney, or the older discussions about SEU vs. EU, evidential vs. causal decision theory, etc.

It's a rather technical book, focusing on issues like advanced sensor data fusion, control theory, and optimization. Not that there is anything wrong with that, the title is just slightly misleading.

For control theory from a mathematician, Fleming, long at the Division of Applied Mathematics at Brown University, there is:

Wendell H. Fleming and Raymond W. Rishel, Deterministic and Stochastic Optimal Control, ISBN 0-387-90155-8, Springer-Verlag, Berlin, 1979.

There is more on optimization, especially from the Hahn-Banach theorem, and also Kalman filtering, in

David G. Luenberger, Optimization by Vector Space Methods, John Wiley and Sons, Inc., New York, 1969.

Can you recommend any good books or other resources?

Ben Recht [0] has worked on control theory and reinforcement learning.

See his blog post "What we've Learned to control" [1] and the survey paper "A Tour of Reinforcement Learning: The View from Continuous Control" [2] may be of interest.

[0]: https://people.eecs.berkeley.edu/~brecht/

[1]: https://www.argmin.net/2020/06/29/tour-revisited/

[2]: https://arxiv.org/abs/1806.09460

I learned it from a course where my professor had his own notes printed by the school bookstore for us, and I've held onto those notes like they're gold. So I can't refer you anything by firsthand knowledge, but the internet seems to like Dorf & Bishop's Modern Control Systems, which seems like a decent coverage of the basics:

https://smile.amazon.com/Modern-Control-Systems-13th-Richard...

After the basic PID controller, what would you consider the most useful tools for control?

I know virtually nothing about control theory, but when I was into RC quadcopters back around 2011 I know Kalman filters got name dropped a lot.

https://en.wikipedia.org/wiki/Kalman_filter

Kalman filters and variations are covered in Ch. 19. It can be viewed as a special case of belief updating in a partially observable Markov decision process.

the next step after PID control is state-space based control.

it's not a single tool but more of a framework. it goes beyond frequency based representations and can also model non-linear control problems.

the kalman filter is an example of the use of this framework where it's combined with statistics.

the problem in control theory though is that once you want to go beyond linear control things get very difficult, most of the literature seems to be about finding clever ways of approximating your problem into something linear.

Check out model predictive control. And LQR if you haven’t learned that yet. And the bazillion state estimation techniques that start with Kalman filtering variants (eg UKF) and progressively account more and more for nonlinearities.

Reply on news.ycombinator.com