Sparse Representations, Inference and Learning
Clarissa Lauditi Emanuele Troiani Marc Mézard
In recent years statistical physics has proven to be a valuable tool to probe
into large dimensional inference problems such as the ones occurring in machine
learning. Statistical physics provides analytical tools to study fundamental
limitations in their solutions and proposes algorithms to solve individual
instances. In these notes, based on the lectures by Marc M\'ezard in 2022 at
the summer school in Les Houches, we will present a general framework that can
be used in a large variety of problems with weak long-range interactions,
including the compressed sensing problem, or the problem of learning in a
perceptron. We shall see how these problems can be studied at the replica
symmetric level, using developments of the cavity methods, both as a
theoretical tool and as an algorithm.