A least distance estimator for a multivariate regression model using
  deep neural networks
Jungmin Shin Seung Jun Shin Sungwan Bang
We propose a deep neural network (DNN) based least distance (LD) estimator
(DNN-LD) for a multivariate regression problem, addressing the limitations of
the conventional methods. Due to the flexibility of a DNN structure, both
linear and nonlinear conditional mean functions can be easily modeled, and a
multivariate regression model can be realized by simply adding extra nodes at
the output layer. The proposed method is more efficient in capturing the
dependency structure among responses than the least squares loss, and robust to
outliers. In addition, we consider $L_1$-type penalization for variable
selection, crucial in analyzing high-dimensional data. Namely, we propose what
we call (A)GDNN-LD estimator that enjoys variable selection and model
estimation simultaneously, by applying the (adaptive) group Lasso penalty to
weight parameters in the DNN structure. For the computation, we propose a
quadratic smoothing approximation method to facilitate optimizing the
non-smooth objective function based on the least distance loss. The simulation
studies and a real data analysis demonstrate the promising performance of the
proposed method.