👨‍👩‍👧‍👦 Exponential Family

The exponential family of distributions is a group of probability distributions that have favorable computation and inference properties. These distributions can also be described with parameters that stay constant with respect to the size of the data.

A member in the family is parameterized in the form

where is a vector of sufficient statistics that define . is a function only on that can be absorbed into the exponent via , and is a normalization constant called the log-partition function that ensures the distribution is valid. The crux of the distribution is described as

where we use the dot product for the 🎳 Inner Product.

Examples §

Common members of the exponential family include 👑 Gaussian, Bernoulli, Binomial, Multinomial, Beta, and Laplace, listed respectively. 1.