News

Virtual reality (VR) and augmented reality (AR) are part of our society and our day to day. Although, portable technology has made great and fast advances in this sector in recent years; there are still many rudimentary components and processes. This is the case of those VR and AR wearables that require, in addition to the visor (glasses), a special command or gloves that detect hand movements; such as, for example, devices intended for video games.

The next step to motion sensors, used in VR and AR devices to detect hand gestures and position, is the heat map.

Research engineers from Cornell University and University of Wisconsin – Madison develop a new motion detection system made of thermal sensors: FingerTrack. Although the technological solution is still in a primitive state, it offers the possibility of perceiving the hands’ movement with a high degree of precision . It is a bracelet – although it currently still has a very cumbersome structure – made up of four thermal cameras about the size of a pea. These cameras capture images of the contour of the hand through the combination of the four images; including the positions of each finger and they send this information to an algorithm powerful enough to be able to make a precise reconstruction of it in real time. The research work was published in the Proceedings of the Association for Computing Machinery on Interactive, Mobile, Wearable and Ubiquitous Technologies.

“This was a major discovery by our team – that by looking at your wrist contours, the technology could reconstruct in 3D, with keen accuracy, where your fingers are. (…) It’s the first system to reconstruct your full hand posture based on the contours of the wrist.”

Cheng Zhang (study author)

Through artificial intelligence, FingerTrak is able to predict up to 20 positions made by the finger joints; and all thanks to the contours of the wrist detected by the heat map.

“How we move our hands and fingers often tells about our health condition, (…) A device like this might be used to better understand how the elderly use their hands in daily life, helping to detect early signs of diseases like Parkinson’s and Alzheimer’s”.

Yin Lin (study author)

There are many other technological solutions on the market designed for the purpose of tracking hand movements; Depth Sensing Cameras, infrared sensors, motion detection gloves, or electromagnetic fingertip sensors, however, all are too bulky for practical use. The FingerTrack developers are optimistic and although the entire usability part of the device still needs to be improved, they assure that all its possibilities will soon be exploited: monitoring and the use of one’s own hands to play video games, remote-controlled robots that reproduce movements of operators, applications in sign language or even monitoring of health problems of a motor nature.