Skip to Main content Skip to Navigation
Journal articles

Affective Interaction with a Virtual Character Through an fNIRS Brain-Computer Interface

G. Aranyi Florian Pecune 1, 2 Fred Charles Catherine Pelachaud 1, 2 Marc Cavazza 
1 MM - Multimédia
LTCI - Laboratoire Traitement et Communication de l'Information
Abstract : Affective brain-computer interfaces (BCI) harness Neuroscience knowledge to develop affective interaction from first principles. In this article, we explore affective engagement with a virtual agent through Neurofeedback (NF). We report an experiment where subjects engage with a virtual agent by expressing positive attitudes towards her under a NF paradigm. We use for affective input the asymmetric activity in the dorsolateral prefrontal cortex (DL-PFC), which has been previously found to be related to the high-level affective-motivational dimension of approach/avoidance. The magnitude of left-asymmetric DL-PFC activity, measured using functional near infrared spectroscopy (fNIRS) and treated as a proxy for approach, is mapped onto a control mechanism for the virtual agent’s facial expressions, in which action units (AUs) are activated through a neuralnetwork.Wecarriedoutanexperimentwith18subjects,whichdemonstratedthat subjects are able to successfully engage with the virtual agent by controlling their mental disposition through NF, and that they perceived the agent’s responses as realistic and consistentwiththeirprojectedmentaldisposition.Thisinteractionparadigmisparticularly relevant in the case of affective BCI as it facilitates the volitional activation of specific areas normally not under conscious control. Overall, our contribution reconciles a model of affect derived from brain metabolic data with an ecologically valid, yet computationally controllable, virtual affective communication environment.
Document type :
Journal articles
Complete list of metadata
Contributor : TelecomParis HAL Connect in order to contact the contributor
Submitted on : Friday, September 13, 2019 - 5:04:50 PM
Last modification on : Wednesday, November 3, 2021 - 6:20:48 AM


  • HAL Id : hal-02287538, version 1


G. Aranyi, Florian Pecune, Fred Charles, Catherine Pelachaud, Marc Cavazza. Affective Interaction with a Virtual Character Through an fNIRS Brain-Computer Interface. Frontiers in Computational Neuroscience, Frontiers, 2016. ⟨hal-02287538⟩



Record views