PID Without a PhD (2016) [pdf]
wescottdesign.com
wescottdesign.com
It does a pretty good job of maintaining system responsiveness and latency when there's sustained memory pressure, at least much better than the simpler hysteresis loops that are commonly used for this sort of thing.
I note that he cites "James Clerk Maxwell: “On Governors”. Proceedings of the Royal Society, #100,1868".[1]
Yes, 1868. Yes, that Maxwell, of Maxwell's Equations. That paper is worth reading. Right there, linear control theory was born. And that's about where the state of the art was until 1950 or so.
Modern control theory is closely tied to machine learning. Except that the control theory people want solutions that don't suddenly do funny stuff for some inputs. The math is way beyond me. Even control theory PhDs have trouble now.
[1] https://www.maths.ed.ac.uk/~v1ranick/papers/maxwell1.pdf
The latter is often studied under the umbrella of optimization theory (convex optimization mostly), but most aspects of it are well understood for most systems we care about (with some notable exceptions, of course). I wouldn’t quite say that “even control theory PhDs have trouble now” as there is plenty of work done on the field and many cases are quite useful (but they certainly have many topics to choose from for their theses!) :)
Enough gain to allow throwing it away with negative feedback to get linearity was a ways off in 1868.
He’s not wrong, but the fundamental problem is that control theory has a very limited view of “correct behavior”: stability around a desired point or trajectory. If you can define your desired system state as a fixed or time varying value that can be determined ahead of time, and you would like to prove that your system stays close to that point for some definition of “close”, then this tool may be for you.
But if you can’t, it’s much harder to see how to apply it. And a lot of software problems can’t be defined in such strictly mathematical terms.
Also, if your system is linear or approximately linear then we have good tools. If your system is really honestly nonlinear it gets more dicey. You have to either hope that somebody has already derived a controller for systems of the class you are using, or you have to do novel numerical analysis in order to derive a controller that will stabilize it.
There are two kinds of problems in the world, problems that are linear, and problems I can't solve.
holds true.
Here's an early paper on it https://arxiv.org/abs/2002.09825. Note that if some of our control theory concepts seem a bit screwy, it's because we come from networking, not control theory :).
I've been at an academic lecture where a process control engineer used the example of air conditioning to explain that when your system is controlled, you have to be extremely careful about correlation and causation. If you're not, you can very easily conclude that because the room is hot whenever the AC is running hard, that the AC running hard makes the room hot. He then showed that biologists were making this mistake with respect to certain biomarkers and cancer.
Process control is interesting even in philosophy. For example, there is a process control theorem which proves that the maximally efficient regulator must have an accurate model of the process being regulated. Therefore, anyone who argues that evolution gives us no reason to expect our brains give us an accurate picture of the world must be ignorant of process control. After all, our brains are regulators of our environment for our survival; therefore the maximally efficient way for our brain to regulate our environment is for our brain to have an accurate model of our environment.
Does this mean evolution is bunk? No, clearly not. In fact, it's evolution's fault. Evolution is path-dependent and only maximizes one thing: reproductive fitness. Your eye, for example, has a blindspot in because the rods and cones are behind the fibers that project to the brain. This is an objectively stupid "design" and the eye would be better without it (as octopus eyes are), but...it's there because it was there in your ancestors, and theirs before them and it was good enough for all of them, and even better than whatever their competitors had at some point. Evolution doesn't improve anything but reproductive success.
So you're correct: environmental physics which have never been survival-relevant will have no selection pressure.
But I think you are very incorrect to say that because our brain ignores the nose part of our eyesight that we have an inaccurate model. A model in process-control-speak means some kind of mathematical description, not the inputs to the controller.
More accurate models presumably have costs (e.g., bigger brains are metabolically expensive). The fitness landscape has local maxima, and they may be hard to move away from (as in the eyes I mentioned above). There are lots of caveats like those.
We know that a lot of internal models are not particularly accurate. The visual system only acquires a tiny bit of high-resolution data at time (with a hole in it[0]) and interpolates the rest. Loss aversion, the gamblers' fallacy, and the rest suggest we're not great at modelling uncertain outcomes. Even mental models of physics differ from reality in some key ways [1].
It's possible that these are still slowly improving, but I think it's more likely that much better models have an unfavorable cost-benenfit profile.
[0] The blind spot is different from the nose. There's a whole in your representation of the world, a few degrees away from the center of gaze. Your brain fills it in in surprising ways. Here's a fun demo that's a big hit with kids too: http://people.whitman.edu/~herbrawt/classes/110/blindspotdem...
[1] Josh Tanenbaum's group has some cool studies looking at people's mental models of physics, which you may find interesting: https://www.pnas.org/content/110/45/18327.short
I’d enjoy hearing more about this. Go on, or point to more info?
But more fundamentally, I believe there is a theorem which asserts that it is in general impossible to extract the open loop behavior of a system (i.e. a model of how the system would respond without a controller) solely from closed loop data (i.e. data collected from the system when the controller is on.)
https://www.nature.com/articles/d41586-018-05719-4
>“The current approach is based on the idea that amyloid-β is bad,” says Perry. “My idea is the opposite.” He suggests that amyloid-β and tau accumulation is actually a protective response to age-related metabolic pressures in the cell.
Not just "squint right." All systems can be represented as a pair of functions that map current state to next state and the observation of current state, in response to some perturbation. Aka state evolution/observation.
F: s_n, x_n -> s_n+1
G: s_n, x_n -> y_n
Abstractions in programming that express this nature cleanly are often the ones that we find the most beautiful, at least for me. For example, an iterator has a void perturbation, F is the next() function, and G is just the observation of the current element (the actual state may be hidden behind a getter of some kind). The abstractions that we build on top of iterators through combinators like map/flatten/fold, etc are then homomorphisms between various iterators by chaining the state evolution/observation functions with other operations.
So if you go one degree higher than an iterator you get a generator, where your perturbation might be non-void. In controls then, you can express your plant and controller (even the "hardware" stages) as one generator composed of smaller ones and combinators across them.
A lot of the math behind this in the general case is kind of heady, at least for me (just an undergrad experience here). There's a lot of ground to be covered in particular understanding for machine learning and other nonlinear dynamical systems where analysis/synthesis of a state-space formulation is currently lacking in formal methods.
Very interesting! Do you have any references where such reformulations are done? Or where control problems are formulated in terms of algebra or category theory?
In controls/linear dynamical systems for example, the canonical forms express it directly [1]. Same goes for anything with a clean state-space representation, like Markov Processes.
The notion of "observation" could also be called "doing something useful with state." For a lot of simple state models, like finite automata, the observation is the identity function.
As for the algebra/category theory, I'm not aware of any formal works that get into it (but also I don't keep up with the literature at that level). It's mentioned by various people (because the work was classified) but Shannon was the first to use topological representations of controls. Mason published these as "signal flow graphs" where multiplication/addition/differentiation/integration operations are represented by the edges and vertices of a directed graph, which he used to derive his famous gain rule [2].
My own interest in the abstract algebra related to this was the fact that the edge of a SFG is a multiplication by a state variable and constant (g * s|z^k) and vertices are summations. SFGs themselves are algebraic structures inside what I understand to be an Abelian group (I'm self taught on this, so forgive terminology/notation).
In the pro-audio niche there's a transform on SFGs called the Topology Preserving Transform and has various derivations (Andy Simper, Will Pirkle, Vadim Zavalishin all have writings on it). It's a fairly manual approach, so about a year ago I wrote an algorithm to do it by reformulating Zavalishin's algebraic derivation [3] as a graph transformation. The interesting bit is that when you look at it that way, the TPT is a homomorphism between Abelian groups (the former in the continuous time, the latter discrete time). At least that's how I understand it.
Anyway, this is mostly my self study. I like the abstraction and it makes composing systems very simple. I'm using these approaches in a Rust crate with some combinators on state evolutions/observations like one would iterators, I find it makes things very composable and elegant [4].
[1] https://www.engr.mun.ca/~millan/Eng6825/canonicals.pdf
[2] https://en.wikipedia.org/wiki/Mason%27s_gain_formula
[3] https://www.native-instruments.com/fileadmin/ni_media/downlo...
If you are assuming the next state depends on the current state, doesn't this work for only Markov system i.e where you can make predictions for the future based solely on its present state?
There's a whole bunch of systems where this doesn't apply, no?
For example, an exponential moving average:
s_n+1 = a x_n + (1 - a) s_n
y_n = s_n
where a, x, s, y in Reals
Conceptually, stateful systems are the notion "where I am going depends on where I am." Markov processes are stateful systems where "where I am going depends on where I am, but not upon where I came from."
I do have all the material from "PID Without a PhD" in there -- just watch the three videos with "PID" in the title.
I also recommend Brian Douglas's channel: https://www.youtube.com/user/ControlLectures if for no other reason than because I haven't posted a new video for three years -- but then, he's been posting on some Matlab channel, so you'll want to start with his and then move to that one.
What would you recommend if the feedback is noisy enough that a derivative term would be mostly noise, but using a simple FIR filter to remove the noise introduces a delay which makes the system less responsive?
I'd use a derivative term with bandlimiting, and I'd accept that if I couldn't get the response I needed using feedback in that case then I'd either investigate using feedforward, or I'd work on changing the sensors on the plant so that they're less noisy.
But like many things, there's no replacement for getting your hands on a real life system and gaining real life intuition.
The best way to learn this? Get/make a test bench with safe-ish motor and encoder system. Play with values and measure at the performance in the system. Increasing the P value gains, makes the system "stiffer" but at the expense of stability. Put your hand on the flywheel and "fight it" safely. You can feel the I (Integral) value winding up against your steady state errors. It's quite magical.
As you start to chase the highest performance for your system, you start to think about alternative control strategies. Eventually you will dream up the concept of feedforward control loop. Try to automate the the picking of PID values, then you start to learn about Ziegler–Nichols tuning method. Gain Scheduling. Non Linear Control theory. It never stops.
(If you do any of this, please make sure the motor/flywheel won't kill you if it goes unstable, and please wire a E-stop that's easy to access when it's unstable)
I avoid integral control because it’s less effective and much harder to deal with than good feedforward. Most of the mechanisms we use (in the FIRST Robotics context) have a feedforward model and good system-identification tools so this works out well.
Tuning is important but when you come to design a control system, you need to look at the overall system - and this is something textbooks don't really tell you. Control systems are all about reacting quickly enough and accurately enough. Delay and poor incoming/outgoing signal quality are the enemies of the control system:
1. The delay from sensor to controller (e.g. SPI bus incurred delay between your gyro sensor and your processor)
2. The bandwidth of each component. If your gyro sensor is only 10hz bandwidth, it'd be difficult to control anything reasonably fast. Note that bandwidth is often given in frequency to 3dB attenuation (half the amplitude) - that's not enough, you need to understand the phase distortion: in other words, as you approach the bandwidth limit, does your signal get distorted in time? because that's very bad for a control system.
3. The resolution of your actuator - if you end up using an on-off actuator, it'd be impossible to do any precision control. Do you want to position a motor accurately? Make sure you use more than 8-bit PWM.
4. Static/dynamic response: things are usually not nicely linear. Things at standstill take a different amount of effort to move than once they are in motion.
My experience has shown time and time again that if you get the system fundamentals correct, implementing and tuning the control loop is reasonably trivial. The secret is having a system that's an order of magnitude faster (higher bandwidth) than what you're trying to control.
You really need to think about control systems in the frequency domain: "what I'm controlling requires the bandwidth of X, I need to sample with with n * X, my processor is this fast, my output signal is that accurate, this is the end to end delay".
Someone once said "any non-linear system becomes linear when you sample it quickly enough". I don't know who said it or whether it's even remotely true for all parts of life - for control systems though, it's definitely a rule to go by.
It's called "PID Without a PhD" but it still assumes at least an undergrad level training.
I wish it had been around when I was studying EE-2-Control. That was not one of my favourite courses but is one of the bits of theory that I have reached for the most since and this guide has been a very useful reminder of much of it.
http://brettbeauregard.com/blog/2011/04/improving-the-beginn...
'PID Control The "PID" in "PID Control" stands for "Proportional, Integral, Derivative". These three terms describe the basic elements of a PID controller.'
But, I had been writing smooth movement and animation for years and never needed a PID controller before, or implemented one accidentally. In fact using animation smoothing is what got me into trouble with the AI driving. In retrospect, I would call basic animation smoothing a “P” controller, you do something in proportion to your goal. To make it smoother, you just turn down the proportion. I tried making the AI driver use a P-controller to steer in order to stay on the optimal racing line.
When you introduce a delay in the system between input and reaction - which is what happens when your car is subject to physics, and when steering changes are rate limited - this is when animation smoothing (P-controller) causes a really surprising behavior. The lower you turn down the P value, the more unstable the system becomes. I was confused at first... why would reducing the input cause the system to go from oscillating to divergent? But when someone suggested reading up on PID controllers, it all made sense, and I was a bit stunned that I hadn’t even heard about them during my undergrad CS years.
The mentioned use of optimal control is one, although if you go there you need to be careful about setting your costs -- LQR and other optimal schemes assume a perfect model of the plant; the higher you set Q the higher the probability that your system will ring or go unstable.
There's various robust control methods, all of which I haven't used in a Good Long While, because for the most part swept-sine measurements work nice.
The one I've used most involves taking swept-sine measurements to get the plant response in Bode plot form, then using either Bode or Nyquist plots (or both) to tune my PID (or whatever controller I'm using).
For a large class of industrial problems, swept-sine measurements won't do, either because measurements need to be undertaken on production lines that are in operation, and the operators get cranky about things like noticeable sinusoidal variations in the product (think aluminum foil or paper), or because even when operated within safe limits, large machines undergoing swept-sine measurements can be downright scary. In such cases one usually ends up using random excitation or steps (often called "bump testing" if you're in an oil refinery or a paper plant) and some sort of system identification step like ARMA.
If you do end up doing testing followed by system ID, you'll most likely get an approximate plant transfer function -- so it's wise to either use a grain of salt when doing your optimal design, or to use some robust design method or other.
It’s also certainly true that a LQR won’t help you if you have no a priori knowledge of your system, but for many of the mechanisms we need to control this isn’t a problem.
Oh, you system model isn’t first order? Now you need an estimator/Kalman filter. Or more sensors. That’s just more complexity for questionable benifit, which is why lqr is beloved by academics [0]. For anything that would be adaquetly controlled with pid, stick with that. After about 2 hours fiddling with the knobs, it’ll be close enough.
This whole idea of optimality is based on bad intuition. Which states do you care about? Why? Is that more valuable than control effort? Why? Who is doing the economic analysis to determine what ultimately costs us more money? In the end, this thing are tuned just like pid: you stop when the step response looks nice. Besides all that, for a lot of systems, and in particular flexible structures with a lot of states, “penalizing state excursions” isn’t really useful intuition to begin with for almost all state space represtations. You are better off with a pid and notch filter.
[0] I decided to complete my PhD in controls so could make statements like that with at least marginal credibility.
Also from my experience you still need to tune the Q and R the same amount that you would P I and D. and I'm not convinced that it is strictly better at all. In industry at least PID absolutely dominates.
It's also important to realize that just because the LQR is derived by solving an optimization problem, that doesn't mean it gives you the best possible gains for whatever you want. You got the optimal controller to minimize a quadratic cost you made up for a linearized approximation of your system.
Iterative design (guess and check) is absolutely still the state of the art for control design, and using an LQR does not escape that.
If I'm missing something in the formal background, though, I'd love to hear it. My own implementation of digital PID controllers have been based on using Ziegler-Nichols for tuning bc I was under the impression above. Any literature or methods contrary to that would definitely be welcome!
(Edited because my first line didn't make any sense)
[1] https://pages.mtu.edu/~tbco/cm416/tuning_methods.pdf
[2] https://en.wikipedia.org/wiki/PID_controller#Loop_tuning
B. Adjust P for 1/4 wave damping , Adjust D to improve response , Adjust I to correct Offset
Repeat B. Repeat B. Repeat B. Repeat B. Repeat B. Repeat B. . . . .
Many processes like temperature controls want to settle, you just have to help them out a bit. Applying a full PID algorithm to every control problem is a bit like using TCP/IP to solve every networking problem. It will work, but you'll often do better with a domain-focused solution.
He's not even doing that to show ads. He's just doing it because, hey, it's not like he wants to read it.
I've implemented this reinforcement learning algorithm in C++ for safely tuning a PID controller on a hardware system with successful results i.e. has been successfully deployed at customer sites for in-situ tuning.
https://github.com/befelix/SafeOpt
http://papers.nips.cc/paper/6692-safe-model-based-reinforcem...
It uses the temperature of the hottest drive to determine if it needs to spin up the fans or not. Works very very well.
The end result is an extremely quiet NAS even with 24 drives.
1. http://georgegillard.com/documents/2-introduction-to-pid-con...
> The server is temporarily unable to service your request due to the site owner reaching his/her bandwidth limit. Please try again later.
For the record -- it's one way to do it, but the code in there was primarily written to be as easy to understand as possible, not to be the World's Best Controller Implementation.
And that's funny/scary. Hopefully those cases still end up being an improvement over not using PID at all?