Abstract
In recent years, self-supervised learning (SSL) frameworks have been extensively applied to sensor-based Human Activity Recognition (HAR) in order to learn deep representations without data annotations. While SSL frameworks reach performance almost comparable to supervised models, studies on interpreting representations learnt by SSL models are limited. Nevertheless, modern explainability methods could help to unravel the differences between SSL and supervised representations: how they are being learnt, what properties of input data they preserve, and when SSL can be chosen over supervised training. In this paper, we aim to analyze deep representations of two recent SSL frameworks, namely SimCLR and VICReg. Specifically, the emphasis is made on (i) comparing the robustness of supervised and SSL models to corruptions in input data; (ii) explaining predictions of deep learning models using saliency maps and highlighting what input channels are mostly used for predicting various activities; (iii) exploring properties encoded in SSL and supervised representations using probing. Extensive experiments on two single-device datasets (MobiAct and UCI-HAR) have shown that self-supervised learning representations are significantly more robust to noise in unseen data compared to supervised models. In contrast, features learnt by the supervised approaches are more homogeneous across subjects and better encode the nature of activities.
Original language | English |
---|---|
Title of host publication | 2023 IEEE International Joint Conference on Biometrics, IJCB 2023 |
Publisher | IEEE |
ISBN (Electronic) | 9798350337266 |
DOIs | |
Publication status | Published - 2023 |
Event | 2023 IEEE International Joint Conference on Biometrics, IJCB 2023 - Ljubljana, Slovenia Duration: 25 Sept 2023 → 28 Sept 2023 https://ijcb2023.ieee-biometrics.org/ |
Publication series
Series | IEEE International Joint Conference on Biometrics (IJCB). Proceedings |
---|
Conference
Conference | 2023 IEEE International Joint Conference on Biometrics, IJCB 2023 |
---|---|
Abbreviated title | IJCB 2023 |
Country/Territory | Slovenia |
City | Ljubljana |
Period | 25/09/23 → 28/09/23 |
Internet address |