Lesion2Vec: Deep Meta Learning for Few-Shot Lesion Recognition in Capsule Endoscopy Video

Effective and rapid detection of lesions in the Gastrointestinal (GI) tract plays a critical role in how fast gastroenterologist can respond to life-threatening diseases. Capsule Endoscopy (CE) has revolutionized traditional endoscopy procedure by allowing gastroenterologists visualize the entire GI tract non-invasively. Once the tiny capsule is swallowed, it captures sequence of images as it is propelled down the GI tract. A single video can last up to 8 h producing between 30,000 and 100,000 images. Automating the detection of frames containing specific lesion in CE video would relieve gastroenterologists of the arduous task of reviewing the entire video before making diagnosis. Convolutional Neural Network (CNN) based models have been very successful in various image classification tasks. However, they suffer excessive parameters, are sample inefficient and rely on very large amount of training data. Deploying a CNN classifier for lesion detection task will require time-to-time fine-tuning to generalize to any unforeseen category. In this paper, we propose a meta-learning framework followed by a few-shot lesion recognition in CE video. Meta-learning framework is designed to establish similarity or dissimilarity between concepts while few-shot learning (FSL) aims to identify new concepts from only a small number of examples. We train a feature extractor to learn a representation for different small bowel lesions using meta-learning. At the testing stage, the category of an unseen sample is predicted from only a few support examples, thereby allowing the model to generalize to a new category that has never been seen before. We demonstrated the efficacy of this method on real patient CE images. We conducted experiments to evaluate the impact of the number of support samples and compared performance across multiple CNN networks. Our experiment showed that this approach performs competitively with baseline models and is effective in few-shot lesion recognition in CE images.