Deep Triplet Networks with Attention for Sensor-based Human Activity Recognition

Bulat Khaertdinov*, Esam Ghaleb, Stylianos Asteriadis

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingAcademicpeer-review

14 Citations (Web of Science)


One of the most significant challenges in Human Activity Recognition using wearable devices is inter-class similarities and subject heterogeneity. These problems lead to the difficulties in constructing robust feature representations that might negatively affect the quality of recognition. This study, for the first time, applies deep triplet networks with various triplet loss functions and mining methods to the Human Activity Recognition task. Moreover, we introduce a novel method for constructing hard triplets by exploiting similarities between subjects performing the same activities using the concept of Hierarchical Triplet Loss. Our deep triplet models are based on the recent state-of-the-art LSTM networks with two attention mechanisms. The extensive experiments conducted in this paper identify important hyperparameters and settings for training deep metric learning models on widely-used open-source Human Activity Recognition datasets. The comparison of the proposed models against the recent benchmark models shows that deep metric learning approach has the potential to improve the quality of recognition. Specifically, at least one of the implemented triplet networks shows the state-of-the-art results for each dataset used in this study, namely PAMAP2, USC-HAD and MHEALTH. Another positive effect of applying deep triplet networks and especially the proposed sampling algorithm is that feature representations are less affected by inter-class similarities and subject heterogeneity issues.
Original languageEnglish
Title of host publication2021 IEEE International Conference on Pervasive Computing and Communications (PerCom)
PublisherIEEE Xplore
Number of pages10
ISBN (Print)978-1-6654-4725-6
Publication statusPublished - 26 Mar 2021
Event2021 IEEE International Conference on Pervasive Computing and Communications (PerCom) - Kassel, Germany
Duration: 22 Mar 202126 Mar 2021


Conference2021 IEEE International Conference on Pervasive Computing and Communications (PerCom)


  • Measurement
  • Training
  • Adaptation models
  • Wearable computers
  • Conferences
  • Activity recognition
  • Benchmark testing

Cite this