X

Research Associate

We invite applications for a Postdoctoral Research Associate in machine learning based in the School of Informatics, University of Edinburgh. The postholder will also be formally affiliated with the EPSRC-funded Hub in Generative AI and work with Drs Siddharth N. and Michael Gutmann as part of the Hub. This is an outstanding opportunity to conduct methodological research at the frontier of machine learning and to collaborate across a vibrant national network of leading universities and industry partners.

Understanding Model Calibration -- A gentle introduction and visual exploration of calibration and the expected calibration error (ECE)

To be considered reliable, a model must be calibrated so that its confidence in each decision closely reflects its true outcome. In this blogpost we'll take a look at the most commonly used definition for calibration and then dive into a frequently used evaluation measure for model calibration. We'll then cover some of the drawbacks of this measure and how these surfaced the need for additional notions of calibration, which require their own new evaluation measures. This post is not intended to be an in-depth dissection of all works on calibration, nor does it focus on how to calibrate models. Instead, it is meant to provide a gentle introduction to the different notions and their evaluation measures as well as to re-highlight some issues with a measure that is still widely used to evaluate calibration.