GMFlow: Learning Optical Flow via Global Matching
2021
Learning-based optical flow estimation has been dominated with the pipeline
of cost volume with convolutions for flow regression, which is inherently
limited to local correlations and thus is hard to address the long-standing
challenge of large displacements. To alleviate this, the state-of-the-art
method, i.e., RAFT, gradually improves the quality of its predictions by
producing a sequence of flow updates via a large number of iterative
refinements, achieving remarkable performance but slowing down the inference
speed. To enable both high accuracy and efficiency optical flow estimation, we
completely revamp the dominating flow regression pipeline by reformulating
optical flow as a global matching problem. Specifically, we propose a GMFlow
framework, which consists of three main components: a customized Transformer
for feature enhancement, a correlation and softmax layer for global feature
matching, and a self-attention layer for flow propagation. Moreover, we further
introduce a refinement step that reuses GMFlow at higher-resolutions for
residual flow prediction. Our new framework outperforms 32-iteration RAFT's
performance on the challenging Sintel benchmark, while using only one
refinement and running faster, offering new possibilities for efficient and
accurate optical flow estimation. Code will be available at
https://github.com/haofeixu/gmflow.
Keywords:
- Correction
- Source
- Cite
- Save
- Machine Reading By IdeaReader
47
References
0
Citations
NaN
KQI