Issues and Objectives
The SENSE project, which focuses on social emotional signals exchanged during an interaction, covers the whole chain of processes, from the signal acquisition (video, audio, neurophysiological) to their exploitation (virtual agent, musical interaction, people interaction) through their interpretation and modelling. Thus, the aim is to bring together key partners of social signal processing in order to break new grounds for the management of multi-modality or the study of temporality within and between partners for example. More specifically, the methodology will integrate the heterogeneous nature of the processed signals that may be conventional such as audio or video, but also neural or physiological with specific sensors. Thus, the considered signals are multimodal with their own dynamics and temporalities and they may influence each other during the interaction. In the case of an interaction with a virtual agent for example, understanding the sequencing of different socio-emotional signals is critical for the analysis and synthesis of realistic behaviours. In other application areas such as musical interaction, we will be able to analyse and describe complex interactions that may involve many people equipped with intelligent sensors. So, all the models developed in this project will be evaluated on realistic scenarios for these different kinds of applications.