Sam Powerhttps://arxiv.org/abs/2306.07262
'Tight skew adjustment to the Laplace approximation in high dimensions'
- Anya Katsevich
Mode-centred Gaussian posterior approximations are a powerful tool for approximate Bayesian inference in the data-rich regime. Relative to approximations based on more global principles, these approximations can often degenerate somewhat in the presence of skewness and asymmetry. This work proposes a simple skewness correction to such approximations, and develops theory which shows that it can offer substantial improvements relative to the baseline approximation.
The Laplace approximation accuracy in high dimensions: a refined analysis and new skew adjustment
In Bayesian inference, making deductions about a parameter of interest requires one to sample from or compute an integral against a posterior distribution. A popular method to make these computations cheaper in high-dimensional settings is to replace the posterior with its Laplace approximation (LA), a Gaussian distribution. In this work, we derive a leading order decomposition of the LA error, a powerful technique to analyze the accuracy of the approximation more precisely than was possible before. It allows us to derive the first ever skew correction to the LA which provably improves its accuracy by an order of magnitude in the high-dimensional regime. Our approach also enables us to prove both tighter upper bounds on the standard LA and the first ever lower bounds in high dimensions. In particular, we prove that $d^2\ll n$ is in general necessary for accuracy of the LA, where $d$ is dimension and $n$ is sample size. Finally, we apply our theory in two example models: a Dirichlet posterior arising from a multinomial observation, and logistic regression with Gaussian design. In the latter setting, we prove high probability bounds on the accuracy of the LA and skew-corrected LA in powers of $d/\sqrt n$ alone.