Deep learning based multi-organ segmentation and metastases segmentation in whole mouse body and the cryo-imaging cancer imaging and therapy analysis platform (CITAP)

We are creating a cancer imaging and therapy analysis platform (CITAP), featuring image analysis/visualization software and multi-spectral cryo-imaging to support innovations in preclinical cancer research. Cryo-imaging repeatedly sections and tiles microscope images of the tissue block face, providing color anatomy and molecular fluorescence 3D microscopic imaging over vast volumes as large as a whole mouse, with single-metastatic-cell sensitivity. We utilized DenseVNet from NiftyNet for multi-organ segmentation on color anatomy images to further analyze major organs. The proposed algorithm was trained/validated/tested on 70/5/4 color anatomy volumes with manually labeled lung, liver, and spleen. The mean Dice similarity coefficient for lung, liver, and spleen in the test set were 0.89±0.01, 0.92±0.01, and 0.83±0.04. We deem Dice coefficient of <0.9 good for analyzing distribution of metastases. To segment GFP-labeled breast cancer metastases in high resolution green fluorescence images, big and small candidates were segmented using marker-based watershed and multi-scale Laplacian of Gaussian filtering followed by Otsu segmentation respectively. A bounding box around each candidate was classified with a 3D convolutional neural network (CNN). In one test mouse with 226 metastases, CNNbased classification and random forest with hand-crafted features achieved sensitivity/specificity of 0.95/0.89 and 0.92/0.82, respectively. DenseVNet-based organ segmentation allows automatic quantification of GFP-labeled metastases in each organ of interest. In the test mouse with 226 metastases, 78 (1 with size <2mm, 21 with size 0.5mm-2mm, and 56 with size <0.5mm) and 24 (1 with size <2mm, 11 with size 0.5-2mm, and 12 with size <0.5mm) were found in the lung and liver respectively.