DeepBackground: Metamorphic testing for Deep-Learning-driven image recognition systems accompanied by Background-Relevance

2021 
Abstract Context: Recently, advances in Deep Learning (DL) have promoted the development of DL-driven image recognition systems in various fields, such as medical treatment, face detection, etc., almost achieving the same level of performance as the human brain. Nevertheless, using DL-driven image recognition systems in these safety-critical domains requires ensuring the accuracy and the stability of these systems. Recent research in this direction mainly focuses on using the image transformations for the overall image to detect the inconsistency of image recognition systems. However, the influence of the image background region ( i . e . , the region of the image other than the target object) on the recognition result of the systems and the robustness evaluation of the systems are not considered. Objective: To evaluate the robustness of DL-driven image recognition systems about image background region changes, this paper introduces DeepBackground, a novel metamorphic testing method for DL-driven image recognition systems. Method: First, we define a new metric, termed Background-Relevance (BRC) to assess the influence degree of the image background region on the recognition result of the image recognition systems. DeepBackground defines a series of domain-specific metamorphic relations (MRs) combined with BRC and automatically generates many follow-up test images based on these MRs. Finally, DeepBackground detects the inconsistency of these systems and evaluates their robustness about image background changes according to BRC. Results: Our empirical validation on 3 commercial image recognition services and 6 popular convolutional neural networks (CNNs) models shows that DeepBackground can not only evaluate the robustness of these image recognition systems about image background changes according to BRC, but also can detect their inconsistent behaviors. Conclusion: DeepBackground is capable of automatically generating high-quality test input images to detect the inconsistency of the image recognition systems, and evaluating the robustness of these systems about image background changes according to BRC.
    • Correction
    • Source
    • Cite
    • Save
    • Machine Reading By IdeaReader
    57
    References
    0
    Citations
    NaN
    KQI
    []