Very much enjoyed this post on "typicality" as described by @sedielem and how our own intuitions about high dimensional space may not align with reality.

The image shows how the probability of the number of heads in a sequence of unfair coin tosses changes accounting for ordering of sequences of events rather than just the statistics of event occurrences.

I also like gentle nudge to the OOD field to test something like typicality as opposed to thresholding likelihoods.

https://benanne.github.io/2020/09/01/typicality.html

Musings on typicality

A summary of my current thoughts on typicality, and its relevance to likelihood-based generative models.

Sander Dieleman
@edhenry @sedielem @paddy This was a really fascinating read! Thanks for sharing it