🪶 Multi-Fairness Unsatisfiability Theorem

Consider some binary categorization of people (for example, yes or no recidivism). For a model that assigns individuals into bins with scores that represent the positive probability, we can define three notions of fairness:

1. Group calibration: scores are calibrated across groups and bins, so

2. Positive balance: average scores for positive people across groups should be the same,

where is the mean predictions for the positive class. 3. Negative balance: symmetric to the positive balance for the negative class,

Intuitively, the first notion requires the score to mean what they're intended—probabilities—for each subgroup; this means that different people with the same score should be treated comparably. The second and third similarly enforce fairness across predictions for the group—scores for true positives and true negatives should be comparable across groups.

Unfortunately, it's impossible to satisfy all three properties. A proof sketch is as follows.

Let be the number of people in group and be the number of positive labels in group . Also, let be the average score for a negative label and be the average score for a positive label.

Assume for contradiction that all fairness definitions hold. Then, for a group , the sum of calibrated predictions should be the number of positive labels. Splitting them into the positive and negative labels, we have

Across two groups and , this draws a system of two lines. These lines are equal if

which forces the base rates across the two groups to be equal. This condition is rarely satisfied in the real world. Alternatively, if the lines aren't equal, they intersect at , implying perfect prediction, which is again not feasible.

Thus, these three constraints can't be simultaneously satisfied in most cases. More details are discussed in Kleinberg et al.