Power-law Dynamic arising from machine learning
Wei Chen Weitao Du Zhi-Ming Ma Qi Meng
We study a kind of new SDE that was arisen from the research on optimization
in machine learning, we call it power-law dynamic because its stationary
distribution cannot have sub-Gaussian tail and obeys power-law. We prove that
the power-law dynamic is ergodic with unique stationary distribution, provided
the learning rate is small enough. We investigate its first exist time. In
particular, we compare the exit times of the (continuous) power-law dynamic and
its discretization. The comparison can help guide machine learning algorithm.