Gaussian Processes Are Not So Fancy
planspace.org
planspace.org
GP's are awesome. Usually, if you've got a few points, your first approach to "fitting a curve through them" would be to choose some parametric form and hope that you're not overfitting with e.g. too high a degree of polynomial. But what if you've got something piecewise, with unpredictable pieces? And what if you don't need a writable parametric form for your curve, but all you need is to answer the question "given my data, what's the probability distribution over the y value at x=5, 6, 7, 8, weighted over all likely curves that might fit my data based on how well they fit?"
Then a GP fit on your data will work wonders for you, as essentially an oracle for those kinds of queries. (And you can then just use the means of those distributions sampled at small intervals, if all you want is to throw a single curve on the screen.) Depending on your choices of priors and kernels, you can have it do magical things.
Great post. A little typo I found was in the covariance equation (copy pasted when you copy-pasted the equation), there's an extra bracket at the end.
I loved the sentiment at the end:
A Gaussian Process might be useful for you. But please don't assume that it is sophisticated just because the language around it often is, or that its results are automatically true just because they have error bars.