♻️ Gibbs Sampling

Gibbs sampling is a 🎯 Markov Chain Monte Carlo technique for approximately sampling from a distribution . If our joint probability is hard to sample but the conditionals are computable, our transition samples one variable conditioned on all others.

Block Gibbs Sampling §

Some are structured with many conditional independences; one example is the intra-layer nodes in a 🚫 Restricted Boltzmann Machine. If this is the case, we can update many variables at once during our transition, a process known as Block Gibbs sampling.

Note

In latent variable models, we often alternate between and .