Benchmarking of deformable image registration for multiple anatomical sites using digital datasets with ground-truth deformation vector fields.

PURPOSE This study aims to evaluate the accuracy of deformable image registration (DIR) algorithms using datasets with different levels of ground-truth deformation vector fields (DVFs) and to investigate the relationship between DVF errors and contour-based metrics. METHODS Nine pairs of digital datasets were generated through contour-controlled deformations based on three anonymized patients' CTs (head-and-neck, thorax/abdomen, and pelvis) with low, medium, and high deformation intensity for each site using ImSimQA software. Image pairs and their associated contours were imported to MIM-Maestro, Raystation, and Velocity systems, followed by DIR and contour propagation. The system generated DVF and propagated contours were compared with the ground-truth data. The correlation between DVF errors and contour-based metrics was evaluated using the Pearson correlation coefficient (r), while their correlation with volumes were calculated using Spearman correlation coefficient (rho). RESULTS The DVF errors increased with increasing the deformation intensity. All DIR algorithms performed well for esophagus, trachea, left femoral, right femoral, and urethral (mean and maximum DVF errors < 2.50 mm and < 4.27 mm, respectively, DSC: 0.93-0.99). Brain, liver, left lung and bladder showed large DVF errors for all three systems (d(max): 2.8-91.90mm). The minimum and maximum DVF errors, conformity index, and Dice similarity coefficient were correlated with volumes (|rho|: 0.41-0.64), especially for very large or small structures (|rho|: 0.64-0.80). Only mean distance to agreement of Raystation and Velocity highly correlated with some indices of DVF errors (r: 0.70-0.78). CONCLUSIONS Most contours-based metrics had no correlation with DVF errors. For adaptive radiotherapy, well-performed contour propagation does not directly indicate accurate dose deformation and summation within each contour (determined by DVF accuracy). Tolerance values for DVF errors should vary as the acceptable accuracy for overall adaptive radiotherapy depends on anatomical site, deformation intensity, organ size, etc. This study provides benchmark tables for evaluating DIR accuracy in various clinical scenarios.