Comparison of automated crater catalogs for Mars from Benedix et al.
  (2020) and Lee and Hogan (2021)
Christopher Lee
Crater mapping using neural networks and other automated methods has
increased recently with automated Crater Detection Algorithms (CDAs) applied to
planetary bodies throughout the solar system. A recent publication by Benedix
et al. (2020) showed high performance at small scales compared to similar
automated CDAs but with a net positive diameter bias in many crater candidates.
I compare the publicly available catalogs from Benedix et al. (2020) and Lee &
Hogan (2021) and show that the reported performance is sensitive to the metrics
used to test the catalogs. I show how the more permissive comparison methods
indicate a higher CDA performance by allowing worse candidate craters to match
ground-truth craters. I show that the Benedix et al. (2020) catalog has a
substantial performance loss with increasing latitude and identify an image
projection issue that might cause this loss. Finally, I suggest future
applications of neural networks in generating large scientific datasets be
validated using secondary networks with independent data sources or training
methods.