Improving Stability in Decision Tree Models
Dimitris Bertsimas Vassilis Digalakis Jr
Owing to their inherently interpretable structure, decision trees are
commonly used in applications where interpretability is essential. Recent work
has focused on improving various aspects of decision trees, including their
predictive power and robustness; however, their instability, albeit
well-documented, has been addressed to a lesser extent. In this paper, we take
a step towards the stabilization of decision tree models through the lens of
real-world health care applications due to the relevance of stability and
interpretability in this space. We introduce a new distance metric for decision
trees and use it to determine a tree's level of stability. We propose a novel
methodology to train stable decision trees and investigate the existence of
trade-offs that are inherent to decision tree models - including between
stability, predictive power, and interpretability. We demonstrate the value of
the proposed methodology through an extensive quantitative and qualitative
analysis of six case studies from real-world health care applications, and we
show that, on average, with a small 4.6% decrease in predictive power, we gain
a significant 38% improvement in the model's stability.