Uncertainty Guided Multi-Scale Attention Network for Raindrop Removal From a Single Image

Raindrops adhered to a glass window or camera lens appear in various blurring degrees and resolutions due to the difference in the degrees of raindrops aggregation. The removal of raindrops from a rainy image remains a challenging task because of the density and diversity of raindrops. The abundant location and blur level information are strong prior guide to the task of raindrop removal. However, existing methods use a binary mask to locate and estimate the raindrop with the value 1 (adhesion of raindrops) and 0 (no adhesion), which ignores the diversity of raindrops. Meanwhile, it is noticed that different scale versions of a rainy image have similar raindrop patterns, which makes it possible to employ such complementary information to represent raindrops. In this work, we first propose a soft mask with the value in [−1,1] indicating the blurring level of the raindrops on the background, and explore the positive effect of the blur degree attribute of raindrops on the task of raindrop removal. Secondly, we explore the multi-scale fusion representation for raindrops based on the deep features of the input multi-scale images. The framework is termed uncertainty guided multi-scale attention network (UMAN). Specifically, we construct a multi-scale pyramid structure and introduce an iterative mechanism to extract blur-level information about raindrops to guide the removal of raindrops at different scales. We further introduce the attention mechanism to fuse the input image with the blur-level information, which will highlight raindrop information and reduce the effects of redundant noise. Our proposed method is extensively evaluated on several benchmark datasets and obtains convincing results.