Natural interaction multimodal analysis: Expressivity analysis towards adaptive and personalized interfaces

Intelligent personalized systems often ignore the affective aspect of human behavior and focus more on tactile cues of the user activity. A complete user modelling, though, should also incorporate cues such as facial expressions, speech prosody and gesture or body posture expressivity features, in order to dynamically profile the user, fusing all available modalities since these qualitative affective cues contain significant information about the user's non verbal behavior and communication. Towards this direction, this work focuses on automatic extraction of gestural and head expressivity features and related statistical processing. The perspective of adopting a common formalization of using expressivity features for a multitude of visual, emotional modalities is explored and grounded through an overview of experiments on appropriate corpora and the corresponding analysis. © 2012 IEEE.