Fair Classification with Noisy Protected Attributes: A Framework with Provable Guarantees

Due to the deployment of classification algorithms in a multitude of applications directly and indirectly affecting people and society, developing methods that are fair with respect to protected attributes such as gender or race is crucial. However, protected attributes in datasets may be inaccurate due to noise in the data collection or if the protected attributes are imputed either in whole or in part. Such inaccuracies can prevent existing fair classification algorithms from achieving their claimed fairness guarantees. Motivated by this, recent works have studied the fair classification problem in which a binary protected attribute is "noisy" (the protected type is flipped with a known fixed probability) by either suggesting optimization using tighter statistical or equalized odds constraints to counter the noise or by identifying conditions under which prior equalized odds post-processing algorithms can handle noisy attributes. We extend the study of noise-tolerant fair classification to a very general setting. Our main contribution is an optimization framework for learning a fair classifier in the presence of noisy perturbations in the protected attributes that can be employed with linear and linear-fractional class of fairness constraints, comes with probabilistic guarantees on accuracy and fairness, and can handle multiple, non-binary protected attributes. Empirically, we show that our framework can be used to attain either statistical rate or false positive rate fairness guarantees with a minimal loss in accuracy, even when the noise corruption is large in two real-world datasets. Prior existing noisy fair classification approaches, on the other hand, either do not always achieve the desired fairness levels or suffer a larger loss in accuracy for guaranteeing high fairness compared to our framework.