Fusion of Texture and Optical Flow Using Convolutional Neural Networks for Gender Classification in Videos

Automatic Gender Classification (AGC) is an essential problem due to its growing demand in commercial applications, including social media and security environments such as the airport. AGC is a well-researched topic both in the field of Computer Vision and Biometrics. In this paper, we propose the use of decision-level fusion for AGC in videos. Our approach does a decision-level fusion of labels obtained from two fine-tuned deep-networks based on a color image and optical-flow image, respectively, based on Resnet-18 architecture. We compare our proposed method with handcrafted features, which includes the concatenation of the Histogram of Optical Flow (HOF) and Histogram of Oriented Gradients (HOG). We compare it with deep-networks, which includes pre-trained & fine-tuned Resnet-18 based on a color image, and pre-trained & fine-tuned Resnet-18 based on optical flow image. Our fusion-based approach considerably outperforms both the handcrafted features, and the deep-networks previously mentioned. Another advantage of our proposed method is that it can work when the visual features are hidden. We used 98 videos from the HMDB51 action recognition dataset, specifically from the cart-wheel action with an almost 50% training, and testing split without validation set. We achieve an overall accuracy of 79.59% with Resnet-18 network architecture with the proposed method, compared to fine-tuned single-stream Resnet-18 for Color-Stream at 65.30%, and Optical-Flow at 55.10% respectively.