Analysing heavy-tail properties of Stochastic Gradient Descent by means
  of Stochastic Recurrence Equations
Ewa Damek Sebastian Mentemeier
In recent works on the theory of machine learning, it has been observed that
heavy tail properties of Stochastic Gradient Descent (SGD) can be studied in
the probabilistic framework of stochastic recursions. In particular,
G\"{u}rb\"{u}zbalaban et al. (arXiv:2006.04740) considered a setup
corresponding to linear regression for which iterations of SGD can be modelled
by a multivariate affine stochastic recursion $X_k=A_k X_{k-1}+B_k$, for
independent and identically distributed pairs $(A_k, B_k)$, where $A_k$ is a
random symmetric matrix and $B_k$ is a random vector. In this work, we will
answer several open questions of the quoted paper and extend their results by
applying the theory of irreducible-proximal (i-p) matrices.