Disparate Impact Diminishes Consumer Trust Even for Advantaged Users

Tim Draws*, Zoltán Szlávik, Benjamin Timmermans, Nava Tintarev, Kush R. Varshney, Michael Hind

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingAcademicpeer-review

1 Citation (Web of Science)
18 Downloads (Pure)

Abstract

Systems aiming to aid consumers in their decision-making (e.g., by implementing persuasive techniques) are more likely to be effective when consumers trust them. However, recent research has demonstrated that the machine learning algorithms that often underlie such technology can act unfairly towards specific groups (e.g., by making more favorable predictions for men than for women). An undesired disparate impact resulting from this kind of algorithmic unfairness could diminish consumer trust and thereby undermine the purpose of the system. We studied this effect by conducting a between-subjects user study investigating how (gender-related) disparate impact affected consumer trust in an app designed to improve consumers' financial decision-making. Our results show that disparate impact decreased consumers' trust in the system and made them less likely to use it. Moreover, we find that trust was affected to the same degree across consumer groups (i.e., advantaged and disadvantaged users) despite both of these consumer groups recognizing their respective levels of personal benefit. Our findings highlight the importance of fairness in consumer-oriented artificial intelligence systems.

Original languageEnglish
Title of host publicationPersuasive Technology
Subtitle of host publication16th International Conference, PERSUASIVE 2021, Virtual Event, April 12–14, 2021, Proceedings
EditorsR. Ali, B. Lugrin, F. Charles
PublisherSpringer, Cham
Pages135-149
Number of pages15
ISBN (Print)978-3-030-79459-0
DOIs
Publication statusPublished - 2021
Event16th International Conference on Persuasive Technologies - Online, Bournemouth University, United Kingdom
Duration: 12 Apr 202114 Apr 2021
https://persuasive2021.bournemouth.ac.uk/

Publication series

SeriesLecture Notes in Computer Science
Volume12684
ISSN0302-9743

Conference

Conference16th International Conference on Persuasive Technologies
Abbreviated titlePERSUASIVE 2021
Country/TerritoryUnited Kingdom
Period12/04/2114/04/21
Internet address

Keywords

  • Disparate impact
  • Algorithmic fairness
  • Consumer trust

Cite this