Computer vision and long short-term memory: Learning to predict unsafe behaviour in construction

Abstract Predicting unsafe behaviour in advance can enable remedial measures to be put in place to mitigate likely accidents on construction sites. Prevailing safety studies in construction tend to be retrospective and focus on examining the conditions that contribute to unsafe behaviour from a psychological perspective. While such studies are warranted, they can also not visually comprehend the dynamic and complex conditions that influence unsafe behaviour. In this paper, we aim to contribute to filling this void and, in doing so, combine computer vision with Long-Short Term Memory (LSTM) to predict unsafe behaviours from videos automatically. Our proposed approach for predicting unsafe behaviour is based on: (1) tracking people using a SiamMask; (2) predicting the trajectory of people using an improved Social-LSTM; and (3) predicting unsafe behaviour using Franklin's point inclusion polygon (PNPoly) algorithm. We use the Wuhan metro project as a case to evaluate our approach’s feasibility and effectiveness. Our adopted SiamMask method outperforms current techniques used for tracking people. Additionally, our improved Social-LSTM can achieve higher accuracy on trajectory prediction than other methods (e.g., Social-GAN). The research findings demonstrate that our developed computer vision approach can be used to accurately predict unsafe behaviour on construction sites.