🕋 Deep Belief Network

Deep Learning / General Models

Info

Deep belief networks were one of the first deep learning successes, beating kernelized SVMs on the MNIST dataset and demonstrating that deep architectures can be successful too.

Deep belief networks are generative models structured as a layer of visible units and layers of latent variables. Connections go between neighboring layers; the deepest two are undirected, and the others and directed toward the visible layer, as shown below.

For hidden layers, we maintain weight matrices and bias vectors, with the first for the visible layer. Similar to 🚫 Restricted Boltzmann Machines, we our probability distributions are:

Content by William Liang, written in Obsidian.
Thank you to all the educators who made these notes possible.