BshapeNet: Object Detection and Instance Segmentation with Bounding Shape Masks

Abstract We propose a modularizable component that can predict the boundary shapes and boxes of an image, along with a new masking scheme for improving object detection and instance segmentation. Specifically, we introduce two types of novel masks: a bounding box (bbox) mask and a bounding shape (bshape) mask. For each of these types, we consider two variants—the “Thick” model and the “Scored” model—both of which have the same morphology but differ in ways that make their boundaries thicker. To evaluate our masks, we design extended frameworks by adding a bshape mask (or a bbox mask) branch to a Faster R-CNN, and call this BshapeNet (or BboxNet). Furthermore, we propose BshapeNet+, a network that combines a bshape mask branch with a Mask R-CNN. Among our various models, BshapeNet+ demonstrates the best performance in both tasks. In addition, BshapeNet+ markedly outperforms the baseline models on MS COCO and Cityscapes and achieves highly competitive results with state-of-the-art models. In particular, the experimental results show that our branch works well on small objects and is easily applicable to various models, such as PANet as well as Faster R-CNN and Mask R-CNN.