A Tolerant Edit Distance for Evaluation and Training of Electron Microscopy Reconstruction Algorithms.

We present a measure to compare the labeling of automatic neuron reconstruction methods against ground truth. This measure, which we call tolerant edit distance (TED), is motivated by two observations: (1) Some errors, like small boundary shifts, are tolerable in practice. Which errors are tolerable is application dependent and should be a parameter of the measure. (2) Non-tolerable errors have to be corrected manually. The time needed to do so should be reflected by the error measure and minimized during training. The TED finds the minimal weighted sum of split and merge errors exceeding a given tolerance criterion, and thus provides a time-to-fix estimate. Our measure works on both isotropic and anisotropic EM datasets, the results are intuitive, and errors can be localized in the volume. We also present a structured learning framework for assignment models for anisotropic neuron reconstruction and show how this framework can be used to minimize the TED on annotated training samples. Evaluated on two publicly available EM-datasets, our method shows consistently higher reconstruction accuracy, even on pre-existing measures, than other current learning methods. Furthermore, we show how an appropriately defined tolerance criterion allows us to train on skeleton (i.e., non-volumetric) annotations, which are much faster to obtain in practice.