Magic Wand: Towards Plug-and-Play Gesture Recognition on Smartwatch

We propose Magic Wand which automatically recognizes 2D gestures (e.g., symbol, circle, polygon, letter) performed by users wearing a smartwatch in real-time manner. Meanwhile, users can freely choose their convenient way to perform those gestures in 3D space. In comparison with existing motion sensor based methods, Magic Wand develops a white-box model which adaptively copes with diverse hardware noises and user habits with almost zero overhead. The key principle behind Magic Wand is to utilize 2D stroke sequence for gesture recognition. Magic Wand defines 8 strokes in a unified 2D plane to represent various gestures. While a user is freely performing gestures in 3D space, Magic Wand collects motion data from accelerometer and gyroscope. Meanwhile, Magic Wand removes various acceleration noises and reduces the dimension of 3D acceleration sequences of user gestures. Moreover, Magic Wand develops stroke sequence extraction and matching methods to timely and accurately recognize gestures. We implement Magic Wand and evaluate its performance with 4 smartwatches and 6 users. The evaluation results show that the median recognition accuracy is 94.0% for a set of 20 gestures. For each gesture, the processing overhead is tens of milliseconds.