IIMR: Intangible Interactions in Mixed Reality

Augmented Reality (AR) has been getting increased traction lately with a plethora of AR-enabled devices on the market ranging from smartphones to commercial Head-Mounted Displays (HMDs). With the rise in AR, a need exists for there to be a more natural and intuitive form of interaction with virtual content that does not interfere with the user experience. Moreover, advanced and sophisticated hardware such as the Leap Motion Controller makes this possible by employing sensors that can track hands accurately in real-time. This paper presents a framework, IIMR (Intangible Interactions in Mixed Reality), that operates over a network and allows for mid-air interactions using just the bare-hands. The aim is to provide researchers and developers with an inexpensive and accessible solution for quickly developing and prototyping hand interactions in a mixed reality environment. The system consists of three modules, namely a Smartphone, a Google cardboard-like HMD, and a Leap Motion Controller. Diverse applications are developed to evaluate the framework, and the results from qualitative usability studies are outlined.

This work was accepted to the ACM International Conference on Interactive Surfaces and Spaces 2020 (ISS'20).

DOI: https://dl.acm.org/doi/abs/10.1145/3380867.3426203

The framework is available at https://github.com/rajshekarreddy97/IIMR