Enabling tabular deep learning when $d \gg n$ with an auxiliary
  knowledge graph
Camilo Ruiz Hongyu Ren Kexin Huang Jure Leskovec
Machine learning models exhibit strong performance on datasets with abundant
labeled samples. However, for tabular datasets with extremely high
$d$-dimensional features but limited $n$ samples (i.e. $d \gg n$), machine
learning models struggle to achieve strong performance due to the risk of
overfitting. Here, our key insight is that there is often abundant, auxiliary
domain information describing input features which can be structured as a
heterogeneous knowledge graph (KG). We propose PLATO, a method that achieves
strong performance on tabular data with $d \gg n$ by using an auxiliary KG
describing input features to regularize a multilayer perceptron (MLP). In
PLATO, each input feature corresponds to a node in the auxiliary KG. In the
MLP's first layer, each input feature also corresponds to a weight vector.
PLATO is based on the inductive bias that two input features corresponding to
similar nodes in the auxiliary KG should have similar weight vectors in the
MLP's first layer. PLATO captures this inductive bias by inferring the weight
vector for each input feature from its corresponding node in the KG via a
trainable message-passing function. Across 6 $d \gg n$ datasets, PLATO
outperforms 13 state-of-the-art baselines by up to 10.19%.