In case anyone else wants to know how the Gaussian process is animated, here are the two links provided in the code:

https://math.stackexchange.com/a/1930880/901583

http://mlss.tuebingen.mpg.de/2013/Hennig_2013_Animating_Samp...

This is very nice. Gaussian process regression didn't click for me until I thought of the data as a partially observed sample of size one from a stochastic process.

Practically, I have a hard time using Gaussian process regression. I find regression with splines to be reasonable and fast, and I don't have to fret about the nugget parameter or the covariance function structure. But I admit GP regression has a beautiful theory.

But there is an equivalence between (smoothing) splines and certain types of Gaussian process models. [1]

[1] http://pages.stat.wisc.edu/~wahba/ftp1/oldie/kw70bayes.pdf

For me the good reason to use gaussian regression is the fact that you get an uncertainty on the output.

The big downside is that it takes expert knowledge (to design a proper kernel) and a solid implementation (to avoid the various numerical problems they can produce) to apply them to practical problem. Most implementation either break down very quickly or are not flexible enough for my taste.

I have a Rust implementation [0] which tries to help with the flexibility aspect but it is still very far from perfect.

[0]: https://github.com/nestordemeure/friedrich

I've found it fairly easy to get high performance GPRs fitted with gpytorch; kernel design is another problem altogether. I think they take more tinkering than an equivalent neural network by way of the Gaussian process prior. In other words, we want a "smooth" function, but how smooth do we really want it?

Yep, uncertainty intervals are definitely easier to get with gp regression.

Nicely done. One can study equations all day long and not get the kind of insight a good interactive visualization provides in just a few minutes.

Well done, I particularly enjoy the rug-plot layout and the “marginalized” densities actually being in the margin of the diagram. \o/

Aligning axes across plots/small multiples is such an underused technique imho!

(Have you considered writing the authors of the Distill article/making a PR to their article? Just wondering; your diagram is already nicely presented on your blog.)

This is awesome! I really wonder if GPs will see an uptick in usage in the future, I've seen a lot of interest in them lately.

Also, thanks for including the source so they can be seen in the browser dev tools, don't know if that gets done enough with interactive visualizations like this.