Past Research
- Past Research
- VR System for Improving Social Skills in Teenagers with Autism
- Robot-mediated Response to Joint Attention System for Children with Autism
- Robotic intervention platform for older adults with mild cognitive impairment and dementia
- Collaborative Virtual Environments
- Social interaction skills tutoring using collaborative Augmented Reality (AR) environment for children with ASD
- Assistive avatar in virtual reality puzzle game to improve eye gaze perception in children with ASD
- Physiology-based Dynamic Difficulty Adjustment Driving Task
- Intelligent Agent
- Physiological Data Processing and Physiology-based Emotion Recognition
- EEG-based Affective Computing
- Cognitive Load Measurement
- IntelliCane: Intelligent Walking Cane System
- Tactile Stimulation Device
- A Haptic Virtual System for Improving Fine Motor Skills in Children with Autism Spectrum Disorders
- Teddy Bear Robot
Personnel:
Lian Zhang, Joshua Wade, Dayi Bian, Jing Fan
Goals/Objectives:
- Fuse multimodal information to measure cognitive load of individuals with ASD during use of a driving simulator
Outline:
A novel virtual reality (VR)-based driving system was developed in order to teach driving skills to adolescents with ASD. The objective of this project is to fuse the multimodal information to measure cognitive load during driving such that driving tasks can be individualized for optimal skill learning. This driving system is capable of gathering eye gaze, electroencephalography, and peripheral physiology data in addition to driving performance data. The collected data was used for systematic feature extraction and classification of cognitive loads based on five well-known machine learning methods. Subsequently, three information fusion schemes—feature level fusion, decision level fusion and hybrid level fusion—were explored. We found that multimodal information fusion can be used to measure cognitive load with high accuracy.
Publications:
- Lian, Zhang, Joshua Wade, Dayi Bian, Jing Fan, Amy Swanson, Amy Weitlauf, Zachary Warren, and Nilanjan Sarkar. “Cognitive load measurement in a Virtual Reality-based Driving System for Autism Intervention.” IEEE Transactions on Affective Computing (2016).
- Zhang, Lian, Joshua Wade, Amy Swanson, Amy Weitlauf, Zachary Warren, and Nilanjan Sarkar. “Cognitive state measurement from eye gaze analysis in an intelligent virtual reality driving system for autism intervention.” In Affective Computing and Intelligent Interaction (ACII), 2015 International Conference on, 2015.
- Zhang, Lian, Joshua Wade, Dayi Bian, Jing Fan, Amy Swanson, Amy Weitlauf, Zachary Warren, and Nilanjan Sarkar. “Multimodal fusion for cognitive load measurement in an adaptive virtual reality driving task for autism intervention.” In International Conference on Universal Access in Human-Computer Interaction, 2015.
- Zhang, Lian, Joshua W. Wade, Dayi Bian, Amy Swanson, Zachary Warren, and Nilanjan Sarkar. “Data Fusion for Difficulty Adjustment in an Adaptive Virtual Reality Game System for Autism Intervention.” In International Conference on Human-Computer Interaction, 2014.