Recurrent Attention Networks for Medical Concept Prediction.

This paper presents the working notes for the CRADLE group's participation in the ImageCLEF2019 medical competition. Our group focused on the concept detection task which challenged participants to approximate the mapping from radiology images to concept labels. Traditionally, such a task is often mod-elled as an image tagging or image retrieval problem. However, we empirically discovered that many concept labels had weak visual connotations; hence, image features alone are insufficient for this task. To this end, we utilize a recurrent neural network architecture which enables our model to capture the relational dependencies among concepts in a label set to supplement visual grounding when their association to image features is weak or unclear. We also exploit soft attention and visual gating mechanisms to enable our network to dynamically regulate “where” and “when” to extract visual data for concept generation.