Skeleton-Based Explainable Bodily Expressed Emotion Recognition Through Graph Convolutional Networks

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingAcademicpeer-review

Abstract

Much of the focus on emotion recognition has gone into the face and voice as expressive channels, whereas bodily expressions of emotions are understudied. Moreover, current studies lack the explainability of computational features of body movements related to emotional expressions. Perceptual research on body parts' movements shows that features related to the arms' movements are correlated the most with human perception of emotions. In this paper, our research aims at presenting an explainable approach for bodily expressed emotion recognition. It utilizes the body joints of the human skeleton, representing them as a graph, which is used in Graph Convolutional Networks (GCNs). We improve the modelling of the GCNs by using spatial attention mechanisms based on body parts, i.e. arms, legs and torso. Our study presents a state-of-the-art explainable approach supported by experimental results on two challenging datasets. Evaluations show that the proposed methodology offers accurate performance and explainable decisions. The methodology demonstrates which body part contributes the most in its inference, showing the significance of arm movements in emotion recognition.
Original languageEnglish
Title of host publication16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021)
PublisherIEEE
Pages1-8
Number of pages8
ISBN (Print)9781665431767
DOIs
Publication statusPublished - 15 Dec 2021
Event2021 16th IEEE International Conference on Automatic Face and Gesture Recognition - Online, Jodhpur, India
Duration: 15 Dec 202118 Dec 2021
http://iab-rubric.org/fg2021/

Publication series

SeriesIEEE International Conference on Automatic Face and Gesture Recognition and Workshops
ISSN2326-5396

Conference

Conference2021 16th IEEE International Conference on Automatic Face and Gesture Recognition
Abbreviated titleFG 2021
Country/TerritoryIndia
CityJodhpur
Period15/12/2118/12/21
Internet address

Cite this