One stage lesion detection based on 3D context convolutional neural networks

Abstract Lesion detection from Computed Tomography (CT) scans is a challenge because non-lesions and true lesions always have similar appearances. Therefore, the performance of mainstream 2D image-based object detection algorithms is not promising since the texture and shape of inner-classes are always different. To detect lesions, we propose a novel deep convolutional feature fusion scheme, 3D Context Feature Fusion (3DCFF). Motivated by state-of-the-art object detection algorithms, we use a one-stage framework, rather than a Region Proposal Network, to extract lesions. In addition, because 3D context provides texture, contour, and shape information that are helpful for generating distinguishable lesion features, 3D context is used as the input for the proposed network. Furthermore, the network adopts a multi-resolution fusion scheme among different scales of feature maps. Results of experiments, conducted with the Deeplesion database, show that the proposed 3DCFF performs better and faster than state-of-the-art algorithms, such as Faster R-CNN, RetinaNet, and 3DCE.