Skip to Main content Skip to Navigation
Journal articles

Early Detection of User Engagement Breakdown in Spontaneous Human-Humanoid Interaction

Atef Ben youssef 1, 2 Chloé Clavel 1, 2 Slim Essid 1, 2 
1 S2A - Signal, Statistique et Apprentissage
LTCI - Laboratoire Traitement et Communication de l'Information
Abstract : This paper presents a supervised classification system for forecasting a potential user engagement breakdown in human-robot interaction. We define engagement breakdown as a failure to successfully complete a predefined interaction scenario, where the user leaves before the expected end. The goal is thus to detect as early as possible such a potential engagement breakdown during the interaction between a human and a humanoid robot. To this end, we exploit a dataset that we have collected in real-world conditions where a set of participants were left to spontaneously engage in an interaction with the robot. The dataset is labeled according to the presence/absence of engagement breakdown. This study investigates the use of a multimodal approach to this problem, where a set of non-verbal features is considered to characterize the users' behavior. The use of combined multimodal features is found to effectively improve the performance of the system. The optimal set of data streams useful for this task is the combination of the distance to the robot, gaze and head motion, as well as facial expressions and speech. We study the time extent over which a user's departure can be anticipated. We find that this ability to anticipate the departure depends on the window during which we observe the user behavior.
Document type :
Journal articles
Complete list of metadata
Contributor : TelecomParis HAL Connect in order to contact the contributor
Submitted on : Friday, September 13, 2019 - 5:37:32 PM
Last modification on : Wednesday, November 3, 2021 - 6:20:44 AM


  • HAL Id : hal-02288043, version 1


Atef Ben youssef, Chloé Clavel, Slim Essid. Early Detection of User Engagement Breakdown in Spontaneous Human-Humanoid Interaction. IEEE Transactions on Affective Computing, Institute of Electrical and Electronics Engineers, 2019. ⟨hal-02288043⟩



Record views