Clinical Melanoma Diagnosis with Artificial Intelligence: Insights from
  a Prospective Multicenter Study
Lukas Heinlein Roman C. Maron Achim Hekler Sarah Haggenmüller Christoph Wies Jochen S. Utikal Friedegund Meier Sarah Hobelsberger Frank F. Gellrich Mildred Sergon Axel Hauschild Lars E. French Lucie Heinzerling Justin G. Schlager Kamran Ghoreschi Max Schlaak Franz J. Hilke Gabriela Poch Sören Korsing Carola Berking Markus V. Heppt Michael Erdmann Sebastian Haferkamp Konstantin Drexler Dirk Schadendorf Wiebke Sondermann Matthias Goebeler Bastian Schilling Eva Krieghoff-Henning Titus J. Brinker
Early detection of melanoma, a potentially lethal type of skin cancer with
high prevalence worldwide, improves patient prognosis. In retrospective
studies, artificial intelligence (AI) has proven to be helpful for enhancing
melanoma detection. However, there are few prospective studies confirming these
promising results. Existing studies are limited by low sample sizes, too
homogenous datasets, or lack of inclusion of rare melanoma subtypes, preventing
a fair and thorough evaluation of AI and its generalizability, a crucial aspect
for its application in the clinical setting. Therefore, we assessed 'All Data
are Ext' (ADAE), an established open-source ensemble algorithm for detecting
melanomas, by comparing its diagnostic accuracy to that of dermatologists on a
prospectively collected, external, heterogeneous test set comprising eight
distinct hospitals, four different camera setups, rare melanoma subtypes, and
special anatomical sites. We advanced the algorithm with real test-time
augmentation (R-TTA, i.e. providing real photographs of lesions taken from
multiple angles and averaging the predictions), and evaluated its
generalization capabilities. Overall, the AI showed higher balanced accuracy
than dermatologists (0.798, 95% confidence interval (CI) 0.779-0.814 vs. 0.781,
95% CI 0.760-0.802; p<0.001), obtaining a higher sensitivity (0.921, 95% CI
0.900- 0.942 vs. 0.734, 95% CI 0.701-0.770; p<0.001) at the cost of a lower
specificity (0.673, 95% CI 0.641-0.702 vs. 0.828, 95% CI 0.804-0.852; p<0.001).
As the algorithm exhibited a significant performance advantage on our
heterogeneous dataset exclusively comprising melanoma-suspicious lesions, AI
may offer the potential to support dermatologists particularly in diagnosing
challenging cases.