Range-Doppler Hand Gesture Recognition Using Deep Residual-3DCNN with Transformer Network

Recently hand gesture recognition via millimeter-wave radar has attracted a lot of research attention for human-computer interaction. Encouraged by the ability of deep learning models in successfully tackling hand gesture recognition tasks, we propose a deep neural network (DNN) model namely, Res3DTENet that aims to classify dynamic hand gestures using the radio frequency (RF) signals. We propose a scheme that improves the convolutional process of 3DCNNs with residual skip connection (Res3D) to emphasize local-global information and enriches the intra-frame spatio-temporal feature representation. A multi-head attention transformer encoder (TE) network has been trained over the spatio-temporal features to refine the inter-frame temporal dependencies of range-Doppler sequences. The experiments are carried out on the publicly available Soli hand gesture data set. Based on our extensive experiments, we show that the proposed network achieves improved gesture recognition accuracy than the state-of-the-art hand gesture recognition methods.