Social interaction skills tutoring using collaborative Augmented Reality (AR) environment for children with ASD

Motivation

The goal of this approach is to develop a collaborative environment in augmented reality for autistic children that can be played on a handheld device. Previous work [1] have shown, that collaborative environments in virtual reality settings increase collaboration and skills. However, in the VR environment people are seated at a static location (desktop) and don’t interact face to face with their peers. A collaborative AR environment would make the interaction between the participants more naturalistic. There conversations will also be closer to reality as the participants would be able see each other and be free to move around. Also, a collaborative game using a hand-held device will also be more economical to implement in home-based therapy sessions by parents as it would require less equipment, thus increasing its reach to resource constrained populations.

Research Objective

Our main aim in this study would be to first build a collaborative augmented reality game environment that can be shared using multiple hand-held devices. Second, would be the ability to objectively measure collaboration in an augmented reality environment. We would like to show that the AR game environment can be used to show an increase in collaborative and social skills in Autistic children.

Outline

The AR application will be able to map indoor environments and render virtual objects at particular physical locations. It will also be able to re-localize a device in an indoor environment using visual spatial features. It will also be able to objectively measure engagement in an AR environment, and use it to promote social collaboration between the users.

References

[1] Zhang, L. et al. Design and Evaluation of a collaborative virtual environment (CoMove) for Autism spectrum disorder intervention, “ACM Transactions on Accessible Computing”, 11(2), 253-278. (2018).