Efficient first-order algorithms for large-scale, non-smooth maximum
  entropy models with application to wildfire science
Gabriel P. Langlois Jatan Buch Jérôme Darbon
Maximum entropy (Maxent) models are a class of statistical models that use
the maximum entropy principle to estimate probability distributions from data.
Due to the size of modern data sets, Maxent models need efficient optimization
algorithms to scale well for big data applications. State-of-the-art algorithms
for Maxent models, however, were not originally designed to handle big data
sets; these algorithms either rely on technical devices that may yield
unreliable numerical results, scale poorly, or require smoothness assumptions
that many practical Maxent models lack. In this paper, we present novel
optimization algorithms that overcome the shortcomings of state-of-the-art
algorithms for training large-scale, non-smooth Maxent models. Our proposed
first-order algorithms leverage the Kullback-Leibler divergence to train
large-scale and non-smooth Maxent models efficiently. For Maxent models with
discrete probability distribution of $n$ elements built from samples, each
containing $m$ features, the stepsize parameters estimation and iterations in
our algorithms scale on the order of $O(mn)$ operations and can be trivially
parallelized. Moreover, the strong $\ell_{1}$ convexity of the
Kullback--Leibler divergence allows for larger stepsize parameters, thereby
speeding up the convergence rate of our algorithms. To illustrate the
efficiency of our novel algorithms, we consider the problem of estimating
probabilities of fire occurrences as a function of ecological features in the
Western US MTBS-Interagency wildfire data set. Our numerical results show that
our algorithms outperform the state of the arts by one order of magnitude and
yield results that agree with physical models of wildfire occurrence and
previous statistical analyses of wildfire drivers.