Reconsidering evaluation practices in modular systems: On the
  propagation of errors in MRI prostate cancer detection
Erlend Sortland Rolfsnes Philip Thangngat Trygve Eftestøl Tobias Nordström Fredrik Jäderling Martin Eklund Alvaro Fernandez-Quilez
Magnetic resonance imaging has evolved as a key component for prostate cancer
(PCa) detection, substantially increasing the radiologist workload. Artificial
intelligence (AI) systems can support radiological assessment by segmenting and
classifying lesions in clinically significant (csPCa) and non-clinically
significant (ncsPCa). Commonly, AI systems for PCa detection involve an
automatic prostate segmentation followed by the lesion detection using the
extracted prostate. However, evaluation reports are typically presented in
terms of detection under the assumption of the availability of a highly
accurate segmentation and an idealistic scenario, omitting the propagation of
errors between modules. For that purpose, we evaluate the effect of two
different segmentation networks (s1 and s2) with heterogeneous performances in
the detection stage and compare it with an idealistic setting (s1:89.90+-2.23
vs 88.97+-3.06 ncsPCa, P<.001, 89.30+-4.07 and 88.12+-2.71 csPCa, P<.001). Our
results depict the relevance of a holistic evaluation, accounting for all the
sub-modules involved in the system.