Marginal likelihood is exhaustive leave-p-out cross-validation
이 글은 로그 주변우도(log marginal likelihood, LML)가 모든 가능한 학습-검증 분할에 대한 평균을 취한 완전한 leave-p-out 교차검증과 동일하다는 점을 수학적으로 증명한다. LML은 베이지안 모델 선택에서 중요한 역할을 하지만, 일반화 성능을 완벽히 대변하지는 못하며, 특히 적은 데이터에 조건부인 경우가 많아 한계가 있다. 다만, 가우시안 프로세스 같은 특정 모델에서는 LML을 효율적으로 계산할 수 있어 실용적이다. 이 연구는 베이지안 모델 선택과 전통적 교차검증 간의 연결고리를 명확히 한다.
https://belko.xyz/posts/lml-and-cross-validation/
#bayesian #marginallikelihood #crossvalidation #modelselection #gaussianprocesses
Cordial congrats to Michel Talagrand on winning this year's Abel Prize, well deserved! His works on bounding #stochastic_process are also of great value for #cosmology ! Imagine we lacked knowledge of bounds on #GaussianProcesses !
Via Alexander Terenin: stochastic gradient descent can be used as an efficient approximate sampling algorithm for Gaussian processes. Looks super cool: https://arxiv.org/abs/2306.11589
Gaussian processes are a powerful framework for quantifying uncertainty and for sequential decision-making but are limited by the requirement of solving linear systems. In general, this has a cubic cost in dataset size and is sensitive to conditioning. We explore stochastic gradient algorithms as a computationally efficient method of approximately solving these linear systems: we develop low-variance optimization objectives for sampling from the posterior and extend these to inducing points. Counterintuitively, stochastic gradient descent often produces accurate predictions, even in cases where it does not converge quickly to the optimum. We explain this through a spectral characterization of the implicit bias from non-convergence. We show that stochastic gradient descent produces predictive distributions close to the true posterior both in regions with sufficient data coverage, and in regions sufficiently far away from the data. Experimentally, stochastic gradient descent achieves state-of-the-art performance on sufficiently large-scale or ill-conditioned regression tasks. Its uncertainty estimates match the performance of significantly more expensive baselines on a large-scale Bayesian optimization task.
Funded (£20,410 4-year tax-free stipend) PhD positions available with me at Imperial College London
#PathIntegrals, #DFT, #CompChem, #MachineLearning, #GaussianProcesses, #JuliaLang, #MonteCarlo, excitement, adventure and really wild things!
Next interview round closing Friday 14th July 2023 .
More details and exemplar projects in this Google doc: https://docs.google.com/document/d/1wiG-T8uqgq_-h-Btu1tecmdrZbK-zS006eB6BKGqFgI/edit?usp=sharing
2023 Frost group PhD adverts Overview / TL;DR Introduction Research Group Motivation / Philosophy Eligibility Funding Notes How to apply Project 1: Simulating charge transfer and recombination at an upconverting organic semiconductor interface Project 2: Modelling mixed electron ion conduction ...
@tylerjburch Yes, I hear you 😓
You've likely already fixed your installation and I'm not sure whether you're using #jupyter, but I found this guide really helpful:
https://pkseeg.com/post/jupyter-venv/
Now, I always warn people to never mess with the base installation of #python on a machine but use (virtual) environments instead.
Good luck with the #GaussianProcesses, I'm going through the #pymc tutorials for it right now 😎
I have, once again, made the strange choice to write a blog. This one is about Gaussian Process and, particularly, about what the Markov property looks like when you don't have a linear notion of time to help you define a past and present.
Like all my GP posts, this one is wildly technical but with an aim towards being somewhat useful. The information here is hard to find unless you want to read a 400 page book translated from Russian
https://dansblog.netlify.app/posts/2023-01-21-markov/markov.html
Watch our own @sethaxen summarize our recent #NeurIPS2022 workshop paper on modeling European #paleoclimate using #GaussianProcesses!
sayzard
Shih Ching Fu
Gravity Grinch
Martin Modrák
Jarvist Moore Frost
Lee Zamparo
Dan
ML ⇌ Science Colaboratory