scientistNov 19, 2023

#science #ScienceDiary

#Astology is saying lots of vague predictions and only focusing on the ones, that by chance, happened (& ignoring the predictions, the majority of predictions, that didn't happen)

Also know as guessing and taking credit for, or believing in, the guessers & belief based predictions that happened by chance (a confirmation bias)

Anyone can predict the roll of a dice or the flip of a coin (sometimes)

#horoscopes #probabilty

Show threadEddMay 23, 2018
So lets multiply this up. Daily exposure to making a 1/20000 mistake leads to a probability of ~1/50 for making that mistake at least once (per year). Over a lifetime of ~80 years we’d have a ~4/5 chance of making that mistake at least once. So yeah it’s a bit of a long-winded way of thinking about it but BE KIND TO ONE ANTOHER, the odds of a fuck-up are *severely* stacked against us. #probabilty #skininthegame #taleb #books
Show thread黒木玄 Gen KurokiOct 19, 2017

#statistics #probabilty

Kullback-Leibler情報量の$$
D(q||p)=G(q||p)-I(q),\\
G(q||p)=-\int q(x)\log p(x)\,dx,\\
I(q)=-\int q(x)\log q(x)\,dx
$$という表示中の汎化損失 $G(q||p)$ とシャノン情報量 $I(q)$ にはそれぞれ「可逆圧縮のサイズ」と「その下限」という「情報量」的な解釈もあります。そのとき、KL情報量は「可逆圧縮のサイズが理想的な下限よりも大きな分」(損失)と解釈できるわけです。

もしかしたら、こちらの方がKullback-Leibler情報量の解釈としては有名かも。その辺のことは、いつも紹介している解説文

https://genkuroki.github.io/documents/20160616KullbackLeibler.pdf

の「10 Mcmillanの不等式と平均符号長」に書いてあります。

しかし、可逆圧縮での損失話と予測分布での損失の話はちょっと違うので、Sanovの定理についても知っておいた方がよいです。