Bayesian neural networks are models that parameterize the distribution of its weights rather than directly optimizing the weights themselves. That is, given training data , rather than finding the fixed parameters , bayesian neural networks model and optimize
In doing so, these models can represent prediction uncertainty by inferring . Intuitively, they can be thought of as an infinite 🎻 Ensemble of standard neural nets.
Optimizing is naturally harder than optimizing directly, and there are a variety of approaches. 🎯 Markov Chain Monte Carlo methods estimate the expectation by performing multiple samples for . Alternatively, the 💼 Evidence Lower Bound framework allows us to parameterize the weight distribution and optimize the ELBO,
