AR Haptics

Personnel: Alexandra Watkins, Tiange (T) Zhang, Ritam Ghosh, Akshith Ullal, Nilanjan Sarkar

Goals/Objectives

1. Demonstrate the ability to instill an illusion of weight for virtual objects in mixed reality using a single-degree-of-freedom kinesthetic feedback device
2. Create an innovative thermal feedback system for realistically representing the thermal properties of virtual objects that provides targeted sensation within an identified contact region

Outline:

Recent years have seen significant advances in Mixed/Augmented Reality (MR/AR) technology and research. Unlike the consumer marketing of previous decades, the commercialization of MR/AR devices has expanded beyond entertainment to include enterprise applications. This has motivated its adoption in a variety of fields including education,  healthcare, and industrial processes, where MR/AR increases the effectiveness of on-site training, information processing, telepresence communication, and remote-local collaboration.

 

An important component of immersive MR/AR collaboration and communication is the realistic perception of the user’s virtual workspace. Providing a means of visuo-haptic feedback instead of purely visual feedback enhances a user’s sense of immersion and realism in a virtual environment. The multi-modal nature of MR/AR poses a unique challenge in incorporating haptic feedback due to the need to interact with both real and virtual objects. Current solutions for haptic feedback in virtual reality rarely prioritize allowing the user to maintain the dexterity needed to grasp, move, or actuate real objects in their workspace, or to allow the user to move freely about their environment. The assumption is that the user is not mobile and will interact solely with the controls of the VR device. As such, the tethered, grounded solutions for VR are not directly transferrable to MR/AR experiences, and there is a need for feedback solutions that meet the requirement for AR/MR users to be mobile and unencumbered.

RASL Haptics involves developing innovative technologies that increase the realism and immersiveness of augmented reality (AR) experiences. Specifically, we are focused on creating devices that provide force and thermal feedback to users when interacting with objects in AR. This will allow users to feel the weight of virtual objects, leading to the ability to compare the heaviness of a real object to a virtual one. Additionally, the application of hot and cold sensations when touching virtual surfaces at varying temperatures will provide users with another modality of information feedback regarding objects in their environment.  The combination of force and thermal feedback will further immerse users in the mixed real/virtual environment of AR applications, enhancing realism and boosting the perceived embodiment of virtual agents.

These technologies are ground-breaking in that they are designed specifically to meet the needs of AR experiences, where users interact with both real and virtual objects and are generally more mobile than in purely virtual reality scenarios. Many existing solutions to haptic feedback in VR are not directly transferrable due to their encumbering designs and lack of mobility. Our goal is to eventually develop wearable, battery-powered devices that apply both force and thermal feedback while still keeping the user’s fingers free to touch and grab objects within their environment.