Multi-Task Learning Via SA-FPN and EJ-Head

As a concise framework, Mask R-CNN achieves promising performance in object detection and instance segmentation. However, there is room for improvement in two aspects. One is that performing multi-task prediction needs more credible feature extraction and multi-scale features integration to handle objects with varied scales. We address this problem by using a novel neck module called SA-FPN (Scale Aware Feature Pyramid Networks), which can accurately help detect and segment the objects of multiple scales. The other is that the isolation between detection and instance segmentation branch exists, causing the gap between training and testing processes. So we propose a unified head module named EJ-Head (Effective Joint Head) to combine two branches into one head, not only realizing the interaction between two tasks, but also enhancing the effectiveness of multi-task learning. Comprehensive experiments on MS-COCO benchmark show that our proposed methods bring noticeable gains for both object detection and instance segmentation.1