https://arxiv.org/abs/2307.07131
'Parallelising Glauber dynamics'
- Holden Lee
Glauber dynamics (also 'Gibbs sampling') is an MCMC technique for sampling from certain classes of probability distribution, often those conforming to a suitable graphical structure. When the underlying graph is sparse, various parallelisation strategies have been proposed, with a goal of accelerating real-time convergence of the algorithm. This work demonstrates that for suitably well-behaved **dense** graphical models, it can still be possible to make effective use of parallelisation strategies.
Parallelising Glauber dynamics
For distributions over discrete product spaces $\prod_{i=1}^n Ω_i'$, Glauber dynamics is a Markov chain that at each step, resamples a random coordinate conditioned on the other coordinates. We show that $k$-Glauber dynamics, which resamples a random subset of $k$ coordinates, mixes $k$ times faster in $χ^2$-divergence, and assuming approximate tensorization of entropy, mixes $k$ times faster in KL-divergence. We apply this to obtain parallel algorithms in two settings: (1) For the Ising model $μ_{J,h}(x)\propto \exp(\frac1 2\left\langle x,Jx \right\rangle + \langle h,x\rangle)$ with $\|J\|<1-c$ (the regime where fast mixing is known), we show that we can implement each step of $\widetilde Θ(n/\|J\|_F)$-Glauber dynamics efficiently with a parallel algorithm, resulting in a parallel algorithm with running time $\widetilde O(\|J\|_F) = \widetilde O(\sqrt n)$. (2) For the mixed $p$-spin model at high enough temperature, we show that with high probability we can implement each step of $\widetilde Θ(\sqrt n)$-Glauber dynamics efficiently and obtain running time $\widetilde O(\sqrt n)$.
