Hand gesture detection using MEMS motion sensors and dynamic time warping algorithm

Abstract

In this project, a wearable device based on micro-electro-mechanical systems (MEMS) motion sensors was developed to translate hand gesture language to speech in real time and helped improve way of communications between hearing and speech impaired community and normal people. A smart glove with three 3-axis accelerometers (LSM303DLHC) and microcontrollers (Arduino Pro Micro) was built to obtain the signal variation for different hand gestures. Three hand gestures with the meaning of “Hello” “Come” and “OK” were selected for the testing. The dynamic time warping (DTW) algorithm was applied to compare the testing signal with the reference signals that had already been recorded and classified in the dataset. Once the testing signal was successfully recognized, its meaning would be converted to speech immediately. All the process was implemented in MATLAB and the accuracy was above 95%. In further research, this method can be developed to recognize much more complicated gesture expressions and translate conversation in real time.