Assistive avatar in virtual reality puzzle game to improve eye gaze perception in children with ASD

Personnel: Ashwaq Amat, Huan Zhao, Amy Swanson, Amy Weitlauf, Zachary Warren, & Nilanjan Sarkar.

Goals/Objectives:

1. Design an eye gaze-based interactive puzzle game for children with ASD
2. Design of a virtual puzzle game that is capable of capturing eye gaze data and tracking children’s performance in real-time during game play
3. Design of an assistive avatar that is capable of providing non-verbal gaze cues to the children when they are playing the puzzle game
4. Design of an assistive avatar that is capable of providing cues and hints based on children performance and when children are unable to respond to gaze cues

Outline:

Gaze sharing plays an important role in children’s early learning and social development. Children diagnosed with ASD and at risk have been known to show diminished eye contact and altered gaze. A virtual reality-based puzzle game was designed for children with ASD to be more aware of gaze cues and improve their ability to interpret these cues. The VR game used finite state machines (FSMs) to control the game sequence. An assistive avatar was designed as a playing partner to provide the children with gaze cues on which puzzle piece to move. The VR game was also designed to use the performance data and gaze data in real-time for the avatar to provide hints and assistance to the children if they were unable to progress during the game play.

Publications:

1. AZ Amat, A Swanson, A Weitlauf, Z Warren, N Sarkar, “Design of an Assistive Avatar in Improving Eye Gaze Perception in Children with ASD During Virtual Interaction“,  International Conference on Universal Access in Human-Computer Interaction, 463-474