Creating Healthy Friction: Determining Stakeholder Requirements of Job Recommendation Explanations

Roan Schellingerhout*, Francesco Barile, Nava Tintarev

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingAcademicpeer-review

Abstract

The increased use of information retrieval in recruitment, primarily through job recommender systems (JRSs), can have a large impact on job seekers, recruiters, and companies. As a result, such systems have been determined to be high-risk in recent legislature. This requires JRSs to be trustworthy and transparent, allowing stakeholders to understand why specific recommendations were made. To fulfill this requirement, the stakeholders’ exact preferences and needs need to be determined. To do so, we evaluated an explainable job recommender system using a realistic, task-based, mixed-design user study (n = 30) in which stakeholders had to make decisions based on the model’s explanations. This mixed-methods evaluation consisted of two objective metrics - correctness and efficiency, along with three subjective metrics - trust, transparency, and usefulness. These metrics were evaluated twice per participant, once using real explanations and once using random explanations. The study included a qualitative analysis following a think-aloud protocol while performing tasks adapted to each stakeholder group. We find that providing stakeholders with real explanations does not significantly improve decision-making speed and accuracy. Our results showed a non-significant trend for the real explanations to outperform the random ones on perceived trust, usefulness, and transparency of the system for all stakeholder types. We determine that stakeholders benefit more from interacting with explanations as decision support capable of providing healthy friction, rather than as previously-assumed persuasive tools.
Original languageEnglish
Title of host publicationRecommender Systems for Human Resources 2024
Subtitle of host publicationProceedings of the 4th Workshop on Recommender Systems for Human Resources (RecSys-in-HR 2024)
PublisherCEUR-WS.org
Volume3788
Publication statusPublished - 11 Oct 2024
Event4th Workshop on Recommender Systems for Human Resources - Bari, Italy
Duration: 14 Oct 202418 Oct 2024

Publication series

SeriesCEUR Workshop Proceedings
Volume3788
ISSN1613-0073

Conference

Conference4th Workshop on Recommender Systems for Human Resources
Abbreviated titleRECSYS-in-HR 2024
Country/TerritoryItaly
CityBari
Period14/10/2418/10/24

Keywords

  • Explainable Recommender Systems
  • Explainable User Interfaces
  • Job recommendation
  • User Studies

Fingerprint

Dive into the research topics of 'Creating Healthy Friction: Determining Stakeholder Requirements of Job Recommendation Explanations'. Together they form a unique fingerprint.

Cite this