A novel chromosome instance segmentation method based on geometry and deep learning

In medicine, any abnormalities in the number of chromosomes or the structure of chromosomes may cause the newborn baby to suffer from genetic diseases, such as Edward syndrome and so on. Chromosome karyotype analysis is the most important and common method for prenatal diagnosis to determine whether a newborn baby has chromosome defects refers to segment chromosome instances from stained cell images and arrange chromosome instances according to their categories. However, due to the non-rigid nature of chromosomes, chromosome instances may overlap and adhere to each other, which makes the task of segmenting chromosome instances time-consuming and error-prone. This paper proposes a novel chromosome instance segmentation method that includes three stages. First, we segment a given stained cell image into several segments using geometric connectivity. Second, a machine learning method is proposed to distinguish chromosome of individual instances and clusters. Finally, a deep learning-based method is applied to separate chromosome instances from clusters. It shows that the proposed method achieves 97.61% instance segmentation accuracy in a hold-out clinical dataset with 162 cell images consisting of 7,452 chromosome instances, which is a promising result in clinical application. The innovation of this work is to combine geometry and deep learning to handle tasks for different stages of chromosome instance segmentation issue. The benefit of this innovation is that it can obtain a much better performance than existing geometric-based methods with a small number of training samples. Meanwhile, the segmentation performance of the proposed method is superior to existing methods fully based on deep learning.