Protecting the privacy of humans in video sequences using a computer vision-based de-identification pipeline

Abstract We propose a computer vision-based de-identification pipeline that enables automated protection of privacy of humans in video sequences through obfuscating their appearance, while preserving the naturalness and utility of the de-identified data. Our pipeline specifically addresses de-identifying soft and non-biometric features, such as clothing, hair, skin color etc., which often remain recognizable when simpler techniques such as blurring are applied. Assuming a surveillance scenario, we combine background subtraction based on Gaussian mixtures with an improved version of the GrabCut algorithm to find and segment pedestrians. De-identification is performed by altering the appearance of the segmented pedestrians through the neural art algorithm that uses the responses of a deep neural network to render the pedestrian images in a different style. Experimental evaluation is performed both by automated classification and through a user study. Results suggest that the proposed pipeline successfully de-identifies a range of hard and soft biometric and non-biometric identifiers, including face, clothing and hair.