Convolutional neural networks for multispectral pedestrian detection

Abstract In this paper, we present a pedestrian detection method by leveraging multispectral images which consist of color and thermal image information. Our method is based on the observation that a multispectral image enables us to overcome inherent limitations for pedestrian detection under challenging situations, e.g., insufficient illumination, small size pedestrian instances and occlusion. In order to detect pedestrian under such conditions, we apply deep convolutional neural networks (CNNs) for effectively combining color and thermal image information in multispectral images. We present a novel multispectral network that is built from the region-based fully convolutional networks (R-FCN) network model. A network-in-network (NIN) is employed to fuse these information across different modalities. Experimental results on KAIST benchmark demonstrate that our method surpasses the baseline method R-FCN and other proposed architectures.