A contrastive triplet network for automatic chest X-ray reporting

between normal and abnormal cases using a triplet network. Specifically, triplets including normal and abnormal cases are first constructed. Then, visual tokens of the chest X-ray are extracted and fed to the Transformer to generate an associated report. During training, comparisons between normal and abnormal cases are conducted via contrasting: 1) the visual embedding of the chest X-ray image encoded by the Transformer encoder, and 2) the semantic embedding of the generated report encoded by a pre-trained textual encoder. Comprehensive experiments on two publicly-available databases have shown the good performance of our method.