Video Deraining Via Temporal Aggregation-and-Guidance

2021 
Learning-based video deraining methods generally integrate temporal correlation within the network. But their non-transparency (i.e., difficult to comprehend how to exploit temporal correlation) seriously limits the development for video deraining. To conquer it, this paper proposes a novel Temporal Aggregation-and-Guidance Network (TAG-Net). Concretely, we define a new temporal ensemble model with set representation by modeling correspondence between rain regions of the current frame and rain-free regions of adjacent frames. Further, we build a TAG-Net that contains: 1) temporal aggregation network derived from the ensemble model, which is with newly-designed self-directed attention acting on video sequences, it automatically learns temporal correlation from multiple adjacent frames to optimize the current frame. 2) temporal guidance network, which aims at eliminating rain streaks in intersected rain regions between the current and adjacent frames to enhance the previously-recovered frame. Extensive evaluations verify that TAG-Net yields the best performance against other advanced methods.
    • Correction
    • Source
    • Cite
    • Save
    • Machine Reading By IdeaReader
    0
    References
    1
    Citations
    NaN
    KQI
    []