Exploiting Field Dependencies for Learning on Categorical Data
Zhibin Li Piotr Koniusz Lu Zhang Daniel Edward Pagendam Peyman Moghadam
Traditional approaches for learning on categorical data underexploit the
dependencies between columns (\aka fields) in a dataset because they rely on
the embedding of data points driven alone by the classification/regression
loss. In contrast, we propose a novel method for learning on categorical data
with the goal of exploiting dependencies between fields. Instead of modelling
statistics of features globally (i.e., by the covariance matrix of features),
we learn a global field dependency matrix that captures dependencies between
fields and then we refine the global field dependency matrix at the
instance-wise level with different weights (so-called local dependency
modelling) w.r.t. each field to improve the modelling of the field
dependencies. Our algorithm exploits the meta-learning paradigm, i.e., the
dependency matrices are refined in the inner loop of the meta-learning
algorithm without the use of labels, whereas the outer loop intertwines the
updates of the embedding matrix (the matrix performing projection) and global
dependency matrix in a supervised fashion (with the use of labels). Our method
is simple yet it outperforms several state-of-the-art methods on six popular
dataset benchmarks. Detailed ablation studies provide additional insights into
our method.