Skip to main content

Robotics and Autonomous Systems Lab
at Vanderbilt

Cognitive Load Measurement

Personnel: Lian Zhang, Joshua Wade, Dayi Bian, Jing Fan

Goals/Objectives:

  1. Fuse multimodal information to measure cognitive load of individuals with ASD during use of a driving simulator

Outline:

A novel virtual reality (VR)-based driving system was developed in order to teach driving skills to adolescents with ASD. The objective of this project is to fuse the multimodal information to measure cognitive load during driving such that driving tasks can be individualized for optimal skill learning. This driving system is capable of gathering eye gaze, electroencephalography, and peripheral physiology data in addition to driving performance data. The collected data was used for systematic feature extraction and classification of cognitive loads based on five well-known machine learning methods. Subsequently, three information fusion schemes—feature level fusion, decision level fusion and hybrid level fusion—were explored. We found that multimodal information fusion can be used to measure cognitive load with high accuracy.

Publications:

  1. Lian, Zhang, Joshua Wade, Dayi Bian, Jing Fan, Amy Swanson, Amy Weitlauf, Zachary Warren, and Nilanjan Sarkar. “Cognitive load measurement in a Virtual Reality-based Driving System for Autism Intervention.” IEEE Transactions on Affective Computing (2016).
  2. Zhang, Lian, Joshua Wade, Amy Swanson, Amy Weitlauf, Zachary Warren, and Nilanjan Sarkar. “Cognitive state measurement from eye gaze analysis in an intelligent virtual reality driving system for autism intervention.” In Affective Computing and Intelligent Interaction (ACII), 2015 International Conference on, 2015.
  3. Zhang, Lian, Joshua Wade, Dayi Bian, Jing Fan, Amy Swanson, Amy Weitlauf, Zachary Warren, and Nilanjan Sarkar. “Multimodal fusion for cognitive load measurement in an adaptive virtual reality driving task for autism intervention.” In International Conference on Universal Access in Human-Computer Interaction, 2015.
  4. Zhang, Lian, Joshua W. Wade, Dayi Bian, Amy Swanson, Zachary Warren, and Nilanjan Sarkar. “Data Fusion for Difficulty Adjustment in an Adaptive Virtual Reality Game System for Autism Intervention.” In International Conference on Human-Computer Interaction, 2014.