PIDs: Creating Stable Control in Games
azeemba.com
azeemba.com
Here’s a decent implementation in Python:
The process of tuning the gains on the different inputs of the controllers, then seeing how the system responded made me think about how this is a similar yet much simpler process to training ML models.
I’ve always been a bit skeptical of how easily one would be able to make changes to something as complex as a large ML model, but looking at it like tuning a very complex PID controller kind of made it seem less like black magic to me for some reason
On one hand I like PID controllers. It's a reusable concept which you can apply to a loot of stuff. On the other hand, in my experience it can be very tricky and cumbersome to tweak them.
In the given example, a PID makes perfect sense. There's a single, static target location you need to approach. Adjust velocity with a PID and you're done.
But what if the target location is not static? Imagine that the character needs to move to your mouse pointer? Do PID's still hold up? How do you handle the I and D term then?
It gets even trickier if you move to 3D. You maybe say, hmm, let's use a rigidbody this time instead of a kinematic body for the player character and use PID's to adjust forces to steer velocity and rotation.
But I never found a good way to do that, because in those other cases the error depends on the goal you need to reach, and if the goal constantly changes, I and D are kinda useless...
Usually in a physical control system (that is driven by a PID controller) your limiting factors are the execution element (how fast can you change the input) and your sensors (the resolution of the transducers/sensors dictate the resolution of what you can achieve). A lot of times the PID controller will have a way that it can autotune itself as it's extremely rare to be able to measure the properties of the physical system precisely and for those properties to not vary with temperature and with wear and tear.
Yes, probably there are all kinds of systems to control and autotune a PID. But that's the problem. In gamedev, PID's get constantly recommended for so many things, but when you play around with them you soon realize, that they are quite hard to get right. So then there's two ways: you dive into control systems or you just lerp the value and call it a day.
My bigger point is that the articles like the one linked here just barely scratch the surface and you cannot really jump into it and get an intuitive understanding by just looking at it. that being said, if you want to learn more you can and they can be applied to a large range of problems.
What you are insinuating is not always correct that if you model stuff you'll get a better result. Sometimes PID is more then enough and in fact better since your model can get arbitrarily complex and you don't have any tools to deal with that behavior. It's a bit snobbish that you need to have math and all the shenanigans. The rest of the manufacturing world disagrees with your premise that is very common in academic circles.
Nobody was insinuating anything. I was pointing out that going past an intuitive understanding may be a harder leap than expected.
It's just a game!
How do you handle the I and D term then?
What if the player can also use a gamepad and go from left to right gradually?
The fundamental problem is that the error accumulates while you try to reach the target. But then the target suddenly changes. Do you just drop the previous error? Do you create an upper bound on the error? Or something else? This is the question no PID post ever answers.
I don't say, that this is impossible, but just that PID's in this context are harder to use than they first appear.
If the derivative is on the error when the setpoint is changed all of the sudden the error is increasing instead of decreasing so it works as desired.
There are many books on control systems that cover different forms of the PID algorithm and what applications they suit.
I prefer the velocity form or double derivative form for the simplicity of managing the integrator and applying real world limits on maximum and minimum outputs and rate of change of output. I basically never need such a fast response that it is worth bothering with D.
A mathematics professor solved the differential equations governing my processes so now I can calculate the exact correct gains for critical damping and simulate the response for different limitations such as input sensor resolution and measurement delays. It is wonderful.
You recalculate the D term every iteration, so it doesn’t matter?
The ugly truth about the I term is that in practice people very often leave it at zero. We sometimes call these controllers PD controllers.
Especialy in a game where you don’t have sticky actuators and miscalibrated sensors this is usually more than enough for hapiness.
PIDs sound cool though. I should try em.
Other than that, there are some tricks to make the loops more stable. Limiting the error can be useful. Limiting the action of the actuator. Smoothing the action of the actuator is very useful. I tend to think of PD loops as damped springs. The P term is like the spring constant, and the D term is like friction or viscosity. The first one determines the speed of reaching the goal, and the second one reduces the oscillations when reaching the goal. I rarely use the I term.
The theory isn't that simple (you can take a course in Control Theory which would cover that) but I think many/most people that use/tune PIDs in the real world just use rules of thumb and standard approaches to tune the different components of the loop. Somewhat more advanced, but still non-theoretical tools, are actually measuring the open loop response which lets you pretty much see the margins you have. Optimizing PIDs is really trying to get the loop to respond quickly without becoming unstable.
Another interesting thing that's used in the real world is feed-forward. That is if you have some estimate of how to perform the function of the PID directly you can just feed that to the output and add the PID in parallel. The motion control analogy of this is called acceleration feed forward. If you know the target acceleration you can basically multiply that by some constant and feed it directly to the output (which is generally current to the motor which is more or less linear to acceleration given a constant load). This dramatically improves the performance of your control system because the PID now just corrects the much smaller residual error.
Why do you think that? the error is the difference between actual and desired at every instant. If you move the desired then you enlarge the error term and PID reacts to close that gap, it might never reach if you are changing the goal too fast but that's something related to the concept of "bandwidth". OTherwise you are quite right and all is OK.
If you have multiple axes then the typical solution is to have n PID loops for n axes. This assumes that the dynamics of the system is decoupled. In english, if I do something in one axis it will not affect the other axes. Then you can tune the PID loops independently. Otherwise, dry math.
https://youtu.be/wkfEZmsQqiA?si=50WWz4kuber56JIU
Brian has great videos on other control theory topics as well.
Just implement the camera positioning and lookat-pointing in the dumbest way possible (including non-continuous, sudden jumps), plug a PID into the position and look-at point, tweak the parameters a bit and voilà, pure magic :)
You could remove the root motion of the animation and control the position of runner in code. This would allow controlling the position/speed easily but to look natural you still have to tweak the animation speed dynamically. Just changing the foot position via IK would not be enough to make the animation look natural. So feels like you are still left with the original problem of tweaking the animation speed.
In that example, they do the character's physics first (just a ball), then match the animation to it. The animation speed is controlled by a "surveyor wheel" approach — go to to 336s in the video for that part.
So, the animation speed is derived from the physics, rather than the physics speed being derived from the animation.
I wouldn't call that "IK" exactly, which would be more for things like matching the foot to individual stair steps as the character walks up them, or such. But I suppose in a broader sense it is, since you're deciding on a physical goal a-priori, then forcing the animation to match that goal.
If there's no error in measurements, sure, a mix of IK and blending seems to be a lot easier to implement.
I am surprised to hear about a PD controller though. In my testing, a PI controller seemed to behave much better than a PD controller. In my research, it seemed a common strategy to drop the D-component completely but I did not see the suggestion to drop the I-component.
Practically speaking, I would recommend encoding as much model information as possible before pulling out the I term: for instance, if you know the velocity of your tracked object and assume an intertidal model, you can just include some of that data in your feed forward rather than the controller, which simplifies things greatly.
One thing that I think is often confused the matter is that P, I, and D all relate to what you measure and what you're controlling. E.g. you might measure velocity or position in a motion control system, and generally your output is going to be current. In a temperature control system your measurement would be temperature and your output is likely going to be current (which influences acceleration, not position or velocity). Ofcourse velocity being the derivative of position means that the meaning of "P" is different. I haven't done PID in a long while but I've always used to ground myself in motion control systems to get a sense what I want to put the loop over.
EDIT: E.g. IIRC it's common in motion control to have a PI loop over the position error and an additional proportional controller over the velocity (and then filters, feed-forward and a bunch of other components ;) ).
I was referring to that part. Ie: if I know more abouty system, how do I include that in my control loop.
E.g. if I want to control the temperature of a room, I might have an idea how much energy is lost due to the gradient between inside and outside temperatures.
I would assume that a PID controller is still suitable for that, but I have never seen novice level resources on that.
If you have a mathematical model for the thing you're controlling you can derive the PID parameters using that model. That's very rarely the case though and even with a model reality almost always differs.
The PID controller doesn't "care" about your knowledge. What it does "care" about is how the system responds to the control signal, specifically with linear systems the gain and phase for the different frequencies. If you're tuning a PID using some heuristics then the process of tuning includes the response of the system you're controlling. So at least intuitively I don't see an easy way to take some partial knowledge of the system and incorporate it into the tuning process. Maybe someone with experience can say something like "using less proportional gain and more integral gain results in better <something> for temperature control systems where there's a lot of energy loss due to gradient" but otherwise there's no easy way to tell IMO. If the integrator doesn't help you're probably going to discover that in the process of tuning anyways.
I think this knowledge is more useful in the context of a feed forward, i.e. if you have a good model of how your system behaves and you incorporate it into the feed forward part of your control loop then the residual error is going to be much smaller and you'll get better performance.