Subject-specific Deep Neural Networks for Count Data with
  High-cardinality Categorical Features
Hangbin Lee Il Do Ha Changha Hwang Youngjo Lee
There is a growing interest in subject-specific predictions using deep neural
networks (DNNs) because real-world data often exhibit correlations, which has
been typically overlooked in traditional DNN frameworks. In this paper, we
propose a novel hierarchical likelihood learning framework for introducing
gamma random effects into the Poisson DNN, so as to improve the prediction
performance by capturing both nonlinear effects of input variables and
subject-specific cluster effects. The proposed method simultaneously yields
maximum likelihood estimators for fixed parameters and best unbiased predictors
for random effects by optimizing a single objective function. This approach
enables a fast end-to-end algorithm for handling clustered count data, which
often involve high-cardinality categorical features. Furthermore,
state-of-the-art network architectures can be easily implemented into the
proposed h-likelihood framework. As an example, we introduce multi-head
attention layer and a sparsemax function, which allows feature selection in
high-dimensional settings. To enhance practical performance and learning
efficiency, we present an adjustment procedure for prediction of random
parameters and a method-of-moments estimator for pretraining of variance
component. Various experiential studies and real data analyses confirm the
advantages of our proposed methods.