On Fixing the Right Problems in Predictive Analytics: AUC Is Not the
  Problem
Ryan S. Baker Nigel Bosch Stephen Hutt Andres F. Zambrano Alex J. Bowers
Recently, ACM FAccT published an article by Kwegyir-Aggrey and colleagues
(2023), critiquing the use of AUC ROC in predictive analytics in several
domains. In this article, we offer a critique of that article. Specifically, we
highlight technical inaccuracies in that paper's comparison of metrics,
mis-specification of the interpretation and goals of AUC ROC, the article's use
of the accuracy metric as a gold standard for comparison to AUC ROC, and the
article's application of critiques solely to AUC ROC for concerns that would
apply to the use of any metric. We conclude with a re-framing of the very valid
concerns raised in that article, and discuss how the use of AUC ROC can remain
a valid and appropriate practice in a well-informed predictive analytics
approach taking those concerns into account. We conclude by discussing the
combined use of multiple metrics, including machine learning bias metrics, and
AUC ROC's place in such an approach. Like broccoli, AUC ROC is healthy, but
also like broccoli, researchers and practitioners in our field shouldn't eat a
diet of only AUC ROC.