Explaining, Analyzing, and Probing Representations of Self-Supervised Learning Models for Sensor-based Human Activity Recognition

Bulat Khaertdinov*, Stylianos Asteriadis

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingAcademicpeer-review

Abstract

In recent years, self-supervised learning (SSL) frameworks have been extensively applied to sensor-based Human Activity Recognition (HAR) in order to learn deep representations without data annotations. While SSL frameworks reach performance almost comparable to supervised models, studies on interpreting representations learnt by SSL models are limited. Nevertheless, modern explainability methods could help to unravel the differences between SSL and supervised representations: how they are being learnt, what properties of input data they preserve, and when SSL can be chosen over supervised training. In this paper, we aim to analyze deep representations of two recent SSL frameworks, namely SimCLR and VICReg. Specifically, the emphasis is made on (i) comparing the robustness of supervised and SSL models to corruptions in input data; (ii) explaining predictions of deep learning models using saliency maps and highlighting what input channels are mostly used for predicting various activities; (iii) exploring properties encoded in SSL and supervised representations using probing. Extensive experiments on two single-device datasets (MobiAct and UCI-HAR) have shown that self-supervised learning representations are significantly more robust to noise in unseen data compared to supervised models. In contrast, features learnt by the supervised approaches are more homogeneous across subjects and better encode the nature of activities.
Original languageEnglish
Title of host publication2023 IEEE International Joint Conference on Biometrics, IJCB 2023
PublisherIEEE
ISBN (Electronic)9798350337266
DOIs
Publication statusPublished - 2023
Event2023 IEEE International Joint Conference on Biometrics, IJCB 2023 - Ljubljana, Slovenia
Duration: 25 Sept 202328 Sept 2023
https://ijcb2023.ieee-biometrics.org/

Publication series

SeriesIEEE International Joint Conference on Biometrics (IJCB). Proceedings

Conference

Conference2023 IEEE International Joint Conference on Biometrics, IJCB 2023
Abbreviated titleIJCB 2023
Country/TerritorySlovenia
CityLjubljana
Period25/09/2328/09/23
Internet address

Fingerprint

Dive into the research topics of 'Explaining, Analyzing, and Probing Representations of Self-Supervised Learning Models for Sensor-based Human Activity Recognition'. Together they form a unique fingerprint.

Cite this