As deep learning-based classifiers are increasingly adopted in real-world applications, the importance of understanding how a particular label is chosen grows. Single decision trees are an example of a simple, interpretable classifier, but are unsuitable for use with complex, high-dimensional data. On the other hand, the variational autoencoder (VAE) is designed to learn a factored, low-dimensional representation of data, but typically encodes high-likelihood data in an intrinsically non-separable way.   We introduce the differentiable decision tree (DDT) as a modular component of deep networks and a simple, differentiable loss function that allows for end-to-end optimization of a deep network to compress high-dimensional data for classification by a single decision tree.   We also explore the power of labeled data in a  supervised VAE (SVAE) with a Gaussian mixture prior, which leverages label information to produce a high-quality generative model with improved bounds on log-likelihood.   We combine the SVAE with the DDT to get our classifier+VAE (C+VAE), which is competitive in both classification error and log-likelihood, despite optimizing both simultaneously and using a very simple encoder/decoder architecture. While deep learning approaches are very effective in many classification problems, interpretability of the classifier (why a particular classification was made) can be very difficult, yet critical for many applications. Decision trees are highly interpretable classifiers, so long as the data is encoded such that the classes can be easily separated. We present a differentiable decision tree (DDT) that we connect to a variational autoencoder (VAE) to learn an embedding of the data that the tree can classify with low expected loss. The expected loss of the DDT is differentiable, so standard gradient-based methods may be applied in training.Since we work in a supervised learning setting, it is natural to exploit the label information when training the VAE. Thus, we employ a supervised VAE (SVAE) that uses a class-specific Gaussian mixture distribution as its prior distribution. We found that the SVAE was very effective in exploiting label information, resulting in improved log-likelihood due to separation of classes in latent space. Further, when we combined SVAE with DDT (yielding our Classifier+VAE, or C+VAE), we got a model that is competitive in both classification error and log-likelihood, despite optimizing both simultaneously and using a very simple encoder/decoder architecture. Further, the resultant decision tree revealed clear semantic meanings in its internal nodes.
We combine differentiable decision trees with supervised variational autoencoders to enhance interpretability of classification.  This paper proposes a hybrid model of a variational autoencoder composed with a differentiable decision tree, and an accompanying training scheme, with experiments demonstrating tree classification performance, neg. log likelihood performance, and latent space interpretability. The paper tries to build an interpretable and accurate classifier via stacking a supervised VAE and a differentiable decision tree