Learning a Fair Distance Function for Situation Testing

Abstract - Situation testing is a method used in life sciences to prove discrimination. The idea is to put similar testers, who only differ in their membership to a protected-by-law group, in the same situation such as applying for a job. If the instances of the protected-by-law group are consistently treated less favorably than their non-protected counterparts, we assume discrimination occurred. Recently, data-driven equivalents of this practice were proposed, based on finding similar instances with significant differences in treatment between the protected and unprotected ones. A crucial and highly non-trivial component in these approaches, however, is finding a suitable distance function to define similarity in the dataset. This distance function should disregard attributes irrelevant for the classification, and weigh the other attributes according to their relevance for the label. Ideally, such a distance function should not be provided by the analyst but should be learned from the data without depending on external resources like Causal Bayesian Networks. In this paper, we show how to solve this problem based on learning a Weighted Euclidean distance function. We demonstrate how this new way of defining distances improves the performance of current situation testing algorithms, especially in the presence of irrelevant attributes.

Short bio - Daphne Lenders is a PhD researcher at the University of Antwerp where she, under the supervision of prof. Toon Calders studies fairness in Machine Learning. Daphne is especially interested in the requirements of fair ML algorithms, not just from a technical-, but also from a legal and usability perspective. Her interest in ethical AI applications already developed in her Masters, where she dedicated her thesis to the topic of explainable AI.