Computer Vision and Deep Learning to Manage Safety in Construction: Matching Images of Unsafe Behavior and Semantic Rules

The determination of people.s unsafe behavior from images in construction has been typically based on hand-made rule approaches, which renders it difficult to identify multiple acts of unsafe behavior within an image and accordingly apply safety rules. This article aims to develop a computer vision and deep learning method that can match images of people's unsafe behavior with semantic safety rules. Our proposed method consists of: 1) image feature representation; 2) safety rule feature representation; and 3) feature fusion similarity whereby unsafe behavior extracted from an image is matched with safety rules. We validate the effectiveness of our method using an image database of people's unsafe behavior from different sites associated with the construction of the Wuhan Metro Project (China). The results of our research explicitly demonstrate that our method is robust and can accurately recognize people's unsafe behavior and the corresponding safety rule that has been contravened. To this end, we suggest that construction organizations can use our method to manage safety better as part of a behavior-based safety strategy and thus prevent accidents.