MeChanical puzzle with Leap

Abstract

Spatial ability as a component of human intelligence has been widely studied for years. Studies have shown that spatial ability is vital to the success of various careers, especially in engineering and science, and can be improved through training. With the potential benefits that virtual reality (VR) technology can bring, researchers have been working on immersing VR technology into spatial training. However, despite the fact that these projects demonstrated the strength of VR technology in spatial training, they did not employ a gesture-based interface so that the natural way of manipulating the puzzle pieces is not preserved. Since eye-to-hand coordination is one of the crucial elements for developing spatial skills , it is important to simulate the real hands-on experience in the VR environment. Furthermore, although there are existing research working on gesture-based 3D modeling and virtual puzzling, the solutions usually require an extensive setup of devices to improve finger tracking's accuracy, which in return arouse portability and accessibility problem. In order to develop a tool for supporting spatial training that can fully utilize the VR environment while eye-to-hand coordination can be immersed into the user's experience, this project presents a 3D mechanical puzzle game employing a gesture-based control. In the view of providing a comparatively portable, easy-to-use and accessible solution for the gesture-based control, this project will be implemented with the Leap Motion Controller. This project focuses on the virtualization of 3D assembly puzzles presented as an application developed with Unity3D. The major mechanisms that incorporate the gesture-based control into the gaming environment are gesture detection and collision detection between objects.