Hierarchical neural network for hand pose estimation

Abstract Hand pose estimation plays an important role in human–computer interaction and augmented reality. Regressing the joints coordinates is a difficult task due to the flexibility of the joint, self-occlusion and so on. In this paper, we propose a novel and simple hierarchical neural network for hand pose estimation. The hand joint coordinates are divided into six parts and each part is regressed in sequence with this hierarchical architecture. This can divide the complex task of regressing all hand joints coordinates into several sub-tasks which can make the estimation more accurate. When regress the joint coordinates of one part, the features of other parts may bring negative influence to this part due to the similarity among the fingers, so we use an interference cancellation operation in our hierarchical architecture. At the time the joint coordinates of one part are regressed, the corresponding features will be removed from the hand global feature to eliminate the interference of this part. The obtained features will be used as input for regressing the joints coordinates of the next part. The ablation study verifies the effectiveness of our hierarchical architecture. The experimental results demonstrate that our method can achieve state-of-the-art or comparable results relative to existing methods on four public hand pose datasets.