Thermal Imagery Based Instance Segmentation for Energy Audit Applications in Buildings

Energy audit in buildings is an essential task for optimal energy management and operations. This paper focuses on a machine learning pipeline to quantify heat loss using 60,000 thermal images in buildings. The images are captured from a small Unmanned Aerial System (sUAS) over the last two years to form a large thermal data repository. Intense efforts are made to annotate multiple sections of the buildings (e.g. windows, doors, ground, facade, trees, and sky). Data augmentation processes are then applied to generate a large comprehensive training data set. Object detection and instance segmentation models such as Mask R-CNN, Fast R-CNN, and Faster R-CNN were trained, and tested. The preliminary results indicate that Mask R-CNN has a larger mean average precision (mAP) of (83%) over R-CNN (51%), Fast R-CNN (62%), and Faster R-CNN (62 %) for a threshold of 50%. The surface temperature values from these thermal images (pixel-by-pixel) were then used in the standard heat transfer coefficient (U-value in BTU/hr/Sq.ft./F) calculations.