Landmark-based multimodal human action recognition

Stylianos Asteriadis, Petros Daras

Research output: Contribution to journalArticleAcademicpeer-review

Abstract

Human activity recognition has received a lot of attention recently, mainly thanks to the advancements in sensing technologies and systems' increasing computational power. However, complexity in human movements, sensing devices' noise and person-specific characteristics impose challenges that still remain to be overcome. In the proposed work, a novel, multi-modal human action recognition method is presented for handling the aforementioned issues. Each action is represented by a basis vector and spectral analysis is performed on an affinity matrix of new action feature vectors. Using modality-dependent kernel regressors for computing the affinity matrix, complexity is reduced and robust low-dimensional representations are achieved. The proposed scheme supports online adaptivity of modalities, in a dynamic fashion, according to their automatically inferred reliability. Evaluation on three publicly available datasets demonstrates the potential of the approach.

Original languageEnglish
Pages (from-to)4505-4521
Number of pages17
JournalMultimedia Tools and Applications
Volume76
Issue number3
DOIs
Publication statusPublished - Feb 2017

Keywords

  • Spectral clustering
  • Human action recognition
  • Multimodal fusion
  • HUMAN MOTION
  • SEQUENCES
  • VIDEO

Cite this