Reconnaissance des changements comportementaux des utilisateurs de réalité virtuelle à l'aide de capteurs de mouvements et de capteurs physiologiques

Abstract

Virtual reality (VR) applications offer a wide variety of experiences, eliciting a range of emotional responses. Certain changes in the user's emotions (affect) might be detectable by merely analyzing the movements of head and hands, using information already available from standard VR hardware. Such detection could eventually enable adapting and customizing user experiences. However, there is very little previous work on affect detection in immersive VR (i.e., using a head-mounted display), and it is unclear how much information can be gleaned from the motion of head and hands versus additional sensing modalities. We instrumented a state-of-the-art immersive VR platform with heart-rate and electrodermal activity sensors and investigated the feasibility of using movement data (of head, hands, pelvis, feet), with and without heart-rate and electrodermal activity sensor data, to classify different behaviors. We conducted two experiments. In the first one, users underwent four conditions in VR: (1) a baseline condition with a simple counting task in a calm environment, (2) a game involving a block-placement motor task in a calm environment, (3) the same game, but now with time limits and stressors designed to induce fast-and-stressed behavior, and (4) the same game but after users were familiar with the task, with time limits but without stressors, designed to induce fast-but-comfortable behavior. In the second experiment, users were asked to complete four blocks of tasks requiring them to hit moving targets. One of the four blocks (in counterbalanced order) corresponded to a stressful condition (S), while the others corresponded to a calm condition (C). For each experiment, ten machine learning algorithms were trained on the data. Our results show that just a few seconds of captured data are sufficient to distinguish all four conditions, with over 75% accuracy using all sensor data, and over 60% accuracy using only movement data. Our work quantifies the difference in accuracy that can be expected for different sensing modalities (head and hand movement versus full body movement versus additional physiological sensors). Our results provide guidance for future designers on the choice of classification algorithm and feature set.

Date	15 Oct 2018
Original language	French
Awarding Institution	École de technologie supérieure
Supervisor	Michael John McGuffin (Supervisor)