Explainable Cross-Topic Stance Detection for Search Results

Tim Draws, Karthikeyan Natesan Ramamurthy, Ioana Baldini Soares, Amit Dhurandhar, Inkit Padhi, Benjamin Timmermans, Nava Tintarev

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingAcademicpeer-review

74 Downloads (Pure)

Abstract

One way to help users navigate debated topics online is to apply stance detection in web search. Automatically identifying whether search results are against, neutral, or in favor could facilitate diversification efforts and support interventions that aim to mitigate cognitive biases. To be truly useful in this context, however, stance detection models not only need to make accurate (cross-topic) predictions but also be sufficiently explainable to users when applied to search results - an issue that is currently unclear. This paper presents a study into the feasibility of using current stance detection approaches to assist users in their web search on debated topics. We train and evaluate 10 stance detection models using a stance-annotated data set of 1204 search results. In a preregistered user study (N = 291), we then investigate the quality of stance detection explanations created using different explainability methods and explanation visualization techniques. The models we implement predict stances of search results across topics with satisfying quality (i.e., similar to the state-of-the-art for other data types). However, our results reveal stark differences in explanation quality (i.e., as measured by users' ability to simulate model predictions and their attitudes towards the explanations) between different models and explainability methods. A qualitative analysis of textual user feedback further reveals potential application areas, user concerns, and improvement suggestions for such explanations. Our findings have important implications for the development of user-centered solutions surrounding web search on debated topics.

Original languageEnglish
Title of host publicationCHIIR'23: Proceedings of the 2023 Conference on Human Information Interaction and Retrieval
Pages221-235
DOIs
Publication statusPublished - 2023
EventCHIIR '23: ACM SIGIR Conference on Human Information Interaction and Retrieval - Austin, United States
Duration: 19 Mar 202325 Mar 2023
https://web.cvent.com/event/98ea9144-44a4-4aa2-90a9-53fa30434f2a/summary

Conference

ConferenceCHIIR '23: ACM SIGIR Conference on Human Information Interaction and Retrieval
Abbreviated titleCHIIR 2023
Country/TerritoryUnited States
CityAustin
Period19/03/2325/03/23
Internet address

Fingerprint

Dive into the research topics of 'Explainable Cross-Topic Stance Detection for Search Results'. Together they form a unique fingerprint.

Cite this