Detecting Spatial Orientation Demands during Virtual Navigation using EEG Brain Sensing

Peer reviewed: 
No, item is not peer reviewed.
Scholarly level: 
Graduate student (Masters)
Final version published as: 

Nguyen-Vo, T., DiPaola, S., & Riecke, B. E. (2017). Detecting Spatial Orientation Demands during Virtual Navigation using EEG Brain Sensing (pp. 1–5). Presented at the ACM SIGPLAN Workshop on Software for Augmented and Virtual Reality (SAVR 2017).

Date created: 
Convolutional neural network
Detection model
Machine learning
Spatial orientation
Virtual reality

This study shows how brain sensing can offer insight to the evaluation of human spatial orientation in virtual reality (VR) and establish a role for electroencephalogram (EEG) in virtual navigation. Research suggests that the evaluation of spatial orientation in VR benefits by goingbeyond performance measures or questionnaires to measurements of the user’s cognitive state. While EEG has emerged as a practical brain sensing technology in cognitive research, spatial orientation tasks often rely on multiple factors (e.g., reference frame used, ability to update simulated rotation, and/or left-right confusion) which may be inaccessible to this measurement. EEG has been shown to correlate with human spatial orientation in previous research. In this paper, we use convolutional neural network (CNN), an advanced technique in machine learning, to train a detection model that can identify moments in which VR users experienced some increase in spatial orientation demands in real-time. Our results demonstrate that we can indeed use machine learning technique to detect such cognitive state of increasing spatial orientation demands in virtual reality research with 96% accurate on average.

Document type: 
Conference presentation
Rights remain with the authors.