Hierarchical confusion matrix for classification performance evaluation
Kevin Riehl Michael Neunteufel Martin Hemberg
In this work we propose a novel concept of a hierarchical confusion matrix,
opening the door for popular confusion matrix based (flat) evaluation measures
from binary classification problems, while considering the peculiarities of
hierarchical classification problems. We develop the concept to a generalized
form and prove its applicability to all types of hierarchical classification
problems including directed acyclic graphs, multi path labelling, and non
mandatory leaf node prediction. Finally, we use measures based on the novel
confusion matrix to evaluate models within a benchmark for three real world
hierarchical classification applications and compare the results to established
evaluation measures. The results outline the reasonability of this approach and
its usefulness to evaluate hierarchical classification problems. The
implementation of hierarchical confusion matrix is available on GitHub.