Multimodal Medical Image Fusion in Cardiovascular Applications

Recent publications in the field of medical image fusion point out the value of multi-modality in diagnosis, pre-surgical planning and surgical intervention. The integration of multiple data sources including medical images from different devices or sensors strongly increases reliability and information content. Successfully fused multi-modal data should not contain any artefacts, not remove any relevant information from the original data and minimize redundancy. Image and data fusion aims at providing supplementary clinical information that is not apparent in the individual images alone. Image and data fusion finds many different applications in the fields of remote sensing, military, biometrics, machine vision and medical imaging. The scientific community has established three levels of fusion rules, namely pixel, feature and decision level. Depending on the application, processing technique or available data each level has its importance and proven significance in medical data processing. Each level provides a set of rules that can be applied. The selection of the fusion operator has a strong impact on the quality of the result. It becomes apparent that the selection of level and technique must vary according to the information that needs to be extracted for a certain application. Each technique has its advantages and disadvantages which have to be carefully evaluated. Based on the availability of multimodal devices, such as ultrasound (US), magnetic resonance imaging (MRI) and computed tomography (CT), different images and data of the same object are obtained. The multiple images, the variety of fusion levels and rules lead to an uncountable number of possible combinations. This makes it very difficult for the user to select the most beneficial solution without losing valuable time and resources. Recent research results show great potential is the development of holistic systems that allow the application of different levels in order to take advantage of the value of each individual processing step to optimize the resulting information. This chapter explains the state of the art in cardiovascular medical image fusion. Multimodal image exploitation in the context of cardiovascular plaque detection is selected as application to illustrate the great potential of a multimodal approach comprising diagnosis as well as pre-surgical planning and the intra-operative process.