FIRE: An Optimization Approach for Fast Interpretable Rule Extraction
Brian Liu Rahul Mazumder
We present FIRE, Fast Interpretable Rule Extraction, an optimization-based
framework to extract a small but useful collection of decision rules from tree
ensembles. FIRE selects sparse representative subsets of rules from tree
ensembles, that are easy for a practitioner to examine. To further enhance the
interpretability of the extracted model, FIRE encourages fusing rules during
selection, so that many of the selected decision rules share common
antecedents. The optimization framework utilizes a fusion regularization
penalty to accomplish this, along with a non-convex sparsity-inducing penalty
to aggressively select rules. Optimization problems in FIRE pose a challenge to
off-the-shelf solvers due to problem scale and the non-convexity of the
penalties. To address this, making use of problem-structure, we develop a
specialized solver based on block coordinate descent principles; our solver
performs up to 40x faster than existing solvers. We show in our experiments
that FIRE outperforms state-of-the-art rule ensemble algorithms at building
sparse rule sets, and can deliver more interpretable models compared to
existing methods.