👟 Total Variation Distance

Total variation distance is a distance measure for two probability distributions. Formally, for distributions and , total variation distance

where is an event in the event space . Intuitively, this is the largest possible difference between probabilities for an event.

Thus, we can also interpret this distance as

where is the sample space.

Finally, by Pinsker’s inequality, we have a relationship with ✂️ KL Divergence: