Global Convergence of Online Identification for Mixed Linear Regression
Yujing Liu Zhixin Liu Lei Guo
Mixed linear regression (MLR) is a powerful model for characterizing
nonlinear relationships by utilizing a mixture of linear regression sub-models.
The identification of MLR is a fundamental problem, where most of the existing
results focus on offline algorithms, rely on independent and identically
distributed (i.i.d) data assumptions, and provide local convergence results
only. This paper investigates the online identification and data clustering
problems for two basic classes of MLRs, by introducing two corresponding new
online identification algorithms based on the expectation-maximization (EM)
principle. It is shown that both algorithms will converge globally without
resorting to the traditional i.i.d data assumptions. The main challenge in our
investigation lies in the fact that the gradient of the maximum likelihood
function does not have a unique zero, and a key step in our analysis is to
establish the stability of the corresponding differential equation in order to
apply the celebrated Ljung's ODE method. It is also shown that the
within-cluster error and the probability that the new data is categorized into
the correct cluster are asymptotically the same as those in the case of known
parameters. Finally, numerical simulations are provided to verify the
effectiveness of our online algorithms.