Predicting social assistance beneficiaries: On the social welfare damage of data biases

Targeting error assessments for social transfers commonly rely on accuracy as a performance metric. This process is typically insensitive to the distributional position of incorrectly classified households. In this paper we develop an extended targeting assessment framework for proxy means tests that accounts for societal sensitivity to targeting errors. We use a social welfare framework to weight targeting errors depending on their position in the welfare distribution and for different levels of societal inequality aversion. While this provides a more comprehensive assessment of targeting performance, we show with two case studies that bias in the data, here in the form of label bias and unstable proxy means testing weights, leads to substantial underestimation of welfare losses that disadvantage some groups more than others.