TomoIR

Hand gesture identification is still one of the most compelling and difficult missions in the world of human-computer interaction, especially in AR/VR technique. To tackle this challenge, many approaches have been explored such as computer-vision-based sensing, electromyography, and ultrasound sensing. In this project, we propose a novel sensing method based on the Infrared (IR) Light. The IR light at 850nm has a strong penetrability through human’s body. If we pose IR light on the wrist, the interior muscle structure change will be shown by IR light as shown in Figure1.

Figure1: Picture taken by the IR camera

At the beginning of the project, we design a 3D printed wristband for placing multiple pairs of IR LED and photodiode (Figure2). Then, we use Teensy board as the ADC interface to collect the data from photodiodes and utilize the machine learning pipeline called T4Train to do the simple gesture recognition based on the data of photodiodes. We achieved a demo through the Rock, Paper and Scissors. CHECK the demo video below :).

Figure2: The illustration of position of IR LED and photodiodes and the 3D-printed wristband.

Currently, we are processing the raw data from photodiodes and the number of IR LED and photodiode on the wristband is quite small. This is the reason why we can only identify a small number of hand gestures with big changes. As mentioned in the title, we will implement the tomography as the feature of the machine learning pipeline and increase the number of pair of IR LED and photodiode in our next step.

Life would be too smooth if it had no rubs in it.