> The Kalman Filter algorithm is a powerful tool for estimating and predicting system states in the presence of uncertainty...
How is this different than any of the ML models?
How is this different than any of the ML models?
Where a KF is really going to kick the pants of a multi-layer perception/neural network is how computationally efficient it is. A KF only takes a couple of matrices of size N^2, where N is the number of variables you’re trying to predict. Compare this to a NN with hundreds/thousands of nodes. Also, the KF is “online learning” in that it “is trained as you go” rather than some other ML models that require upfront training, a KF is very useful for live update-and-predict use cases. And, again, it’s extremely computationally efficient, and can run easily on embedded systems. The book ad, here, suggests tracking: so tracking an airplane with an air traffic control radar would be an effective use for a KF. (Where NNets have found any other uses I’m sure you’re aware of.)
Another huge benefit of a KF is that, unlike NNets, KFs are “explainable”, and in fact extremely well understood by many professionals. This means that a KF can be better tuned to suit a purpose with less fear of unexpected results that may be more common in other ML models. Like, KF(S, x) will always return an explainable new state, where NN(x) may result in a surprise state and no amount of analysis can reveal why (and require training a new model, the “retrain and pray” solution).
There’s a couple of differences for you.
A lot of technical decisions aren't based on "what's the quickest, cheapest and easiest solution to the problem?" but "what solution is most likely to get me hired at a pay bump when I jump ship?"
Kalman Filters are used in the context of a very specific model-type for dynamic systems (a state-space model, see below) to update states (xₖ) using feedback data from sensors (yₖ). These state-space models can either be derived by fitting data, or they can be derived from first principles through physics equations.
xₖ₊₁ = f(xₖ) + g(uₖ)
yₖ = h(xₖ)
For instance, when driving a car, examples of states (x) are position/velocity/acceleration (which might not be directly measured with a sensor! But can be backed out from a mathematical model from quantities that are measured), sensor measurements (y) might be speedometer, accelerometer readings, and control actions (u) might be throttle position, brake pressure, steering angle. The Kalman filter has a model relating all this in time, and based on that model and sensor readings, it reconstructs/infers the likeliest states in the presence of even noisy measurements. This is why Kalman filters are known as "state estimation" algorithms.
ML models typically do not do this -- they only predict. Kalman filters predict and update.
You basically need to know some kind of a model for the system to run KF. Whereas ML is all about working out the model automatically.
As for similarities, KF is a really efficient implementation of Bayesian inference. I think that any ML model that isn't fundamentally using Bayesian inference, is fundamentally flawed.
ML requires training, significant amounts of compute power, and large datasets.
The Apollo program used Kalman filters with limited compute resources.
Kalman filters are for predicting system states in the presence of uncertainty; ML is really searching for and matching patterns, under uncertainty not in its training set, it tends to to find the glitch in the matrix.
In other words it performs well for certain applications specifically because it allows you to bring in domain knowledge in the form of the process model and known uncertainties. Whereas deep learning models try to generalize the model and learn implicit structure from data.
The model is updated sequentially (online learning).
https://en.wikipedia.org/wiki/Linear%E2%80%93quadratic%E2%80...
Actually in theory you can replace everything with it. So what's the point of asking this question here? Ask it for everything.
Kalman filters, and other similar digital filtering and prediction algorithms, are like scalpels compared to the broadsword of NNs and such. There are plenty of things that you can't or shouldn't use a kalman filter for, but for the tasks that it is suited for, you cannot do better with another solution. ML is mostly hand wavy bullshit, and DSP algorithms are like... doing real math, real engineering.