SwitchTab: Switched Autoencoders Are Effective Tabular Learners
Jing Wu Suiyao Chen Qi Zhao Renat Sergazinov Chen Li Shengjie Liu Chongchao Zhao Tianpei Xie Hanqing Guo Cheng Ji Daniel Cociorva Hakan Brunzel
Self-supervised representation learning methods have achieved significant
success in computer vision and natural language processing, where data samples
exhibit explicit spatial or semantic dependencies. However, applying these
methods to tabular data is challenging due to the less pronounced dependencies
among data samples. In this paper, we address this limitation by introducing
SwitchTab, a novel self-supervised method specifically designed to capture
latent dependencies in tabular data. SwitchTab leverages an asymmetric
encoder-decoder framework to decouple mutual and salient features among data
pairs, resulting in more representative embeddings. These embeddings, in turn,
contribute to better decision boundaries and lead to improved results in
downstream tasks. To validate the effectiveness of SwitchTab, we conduct
extensive experiments across various domains involving tabular data. The
results showcase superior performance in end-to-end prediction tasks with
fine-tuning. Moreover, we demonstrate that pre-trained salient embeddings can
be utilized as plug-and-play features to enhance the performance of various
traditional classification methods (e.g., Logistic Regression, XGBoost, etc.).
Lastly, we highlight the capability of SwitchTab to create explainable
representations through visualization of decoupled mutual and salient features
in the latent space.