Deep Dilation on Multimodality Time Series for Human Activity Recognition

Providing accurate information on people’s activities and behaviors plays an important role in innumerable applications, such as medical, security, and entertainment. In recent years, deep learning has been applied in human activity recognition, and achieved a better performance. However, if the spatial dependency of inter-sensors is considered, it is possible to enhance the discriminative ability. In this paper, we present a novel deep learning framework for human activity recognition problems. First, on the basis of our previous work, we utilize dilated convolutional neural network to extract features of inter-sensors and intra-sensors. Since the extracted features are local and short-temporal, it is necessary to utilize RNN to model the long-temporal dependencies. However, duo to that LSTM and GRU often rely on the completely previous computations, it will result in slow inference and hard convergence. Hence, inspired by the idea of dilation operation, we present a novel recurrent model to learn the temporal dependencies at different time scales. Then, at the topmost layer, a fully-connected layer with softmax function is utilized to generate a class probability distribution, and the predicted activity is obtained. Eventually, we evaluate the proposed framework in two open human activity datasets, OPPORTUNITY and PAMAP2. Results demonstrate that the proposed framework achieves a higher classification performance than the state-of-the-art methods. Moreover, it takes the least time to recognize an activity. Besides, it also performs faster and easier to converge in the training stage.