This may seem like a little niche GP paper. But linear algebra / least squares is everywhere. If you agree that each least-squares fit should be a GP, then every run of a linear solver should also be a GP. For all the same reasons.

It’s also an example for how computationally self-aware agents should work: Keeps track not just of the fact you've only seen finite data so far, but also only done a subset of the necessary computation for training so far. We should want that everywhere.

It turns out you can actually keep track of the error of the linear solver with minimal overhead (just like you can keep track of the GP covariance in addition to the posterior mean, at minimal overhead). And the way to do it applies pretty much across all LinAlg Methods.

(Caveat: one can keep track of the _combined_ error of finite data and finite compute. That’s what you actually want, right? Tracking either of them individually costs extra. I’ll leave this 🔫 lying right here on the table…)

That’s _exactly_ like the data itself: If you have N data, but only use M<N, the posterior mean is “off”.

For the data, we all agree how to deal with this: That’s what the posterior covariance is for, right? Finite data -> finite error, modelled by the covariance.

Why not do the same for compute? Finite compute -> finite error, modelled by … well … the covariance!

Say you have a set of N data. To do GP inference, you call a linear algebra method: Cholesky, Conjugate Gradients, gradient descent (don’t!), inducing points. All these algorithms — even Cholesky! — iterate over an atomic step that amounts to a linear projection (mat-vec product)

You can stop the algorithm before it converges (unless N<1e4, you probably do). If you do, its estimate for the GP (posterior mean _and_ covariance) are not yet right. They have an error.

Oh, and there’ll be socials, too! With the wonderful crowd of students and colleagues in Tübingen.

Particular thanks to the Excellence Cluster ML4Science for financial support!

Sign up now, spaces are filling up fast :).