Temporal Triplet Mining for Personality Recognition

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingAcademicpeer-review

Abstract

One of the primary goals of personality computing is to enhance the automatic understanding of human behavior, making use of various sensing technologies. Recent studies have started to correlate personality patterns described by psychologists with data findings, however, given the subtle delineations of human behaviors, results are specific to predefined contexts. In this paper, we propose a framework for automatic personality recognition that is able to embed different behavioral dynamics evoked by diverse real world scenarios. Specifically, motion features are designed to encode local motion dynamics from the human body, and interpersonal distance (proxemics) features are designed to encode global dynamics in the scene. By using a Convolutional Neural Network (CNN) architecture which utilizes a triplet loss deep metric learning, we learn temporal, as well as discriminative spatio-temporal streams of embeddings to represent patterns of personality behaviors. We experimentally show that the proposed Temporal Triplet Mining strategy leverages the similarity between temporally related samples and, therefore, helps to encode higher semantic movements or sub-movements which are easier to map onto personality labels. Our experiments show that the generated embeddings improve the state-of-the-art results of personality recognition on two public datasets, recorded in different scenarios.
Original languageEnglish
Title of host publication2020 The 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG2020)
Pages171-178
Volume1
DOIs
Publication statusPublished - 2020
Event2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020) - Buenos Aires, Argentina
Duration: 16 Nov 202020 Nov 2020

Conference

Conference2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020)
Abbreviated titleFG 2020
CountryArgentina
CityBuenos Aires
Period16/11/2020/11/20

Cite this