Semi-Bandit Learning for Monotone Stochastic Optimization
Arpit Agarwal Rohan Ghuge Viswanath Nagarajan
Stochastic optimization is a widely used approach for optimization under
uncertainty, where uncertain input parameters are modeled by random variables.
Exact or approximation algorithms have been obtained for several fundamental
problems in this area. However, a significant limitation of this approach is
that it requires full knowledge of the underlying probability distributions.
Can we still get good (approximation) algorithms if these distributions are
unknown, and the algorithm needs to learn them through repeated interactions?
In this paper, we resolve this question for a large class of "monotone"
stochastic problems, by providing a generic online learning algorithm with
$\sqrt{T \log T}$ regret relative to the best approximation algorithm (under
known distributions). Importantly, our online algorithm works in a semi-bandit
setting, where in each period, the algorithm only observes samples from the
r.v.s that were actually probed. Our framework applies to several fundamental
problems in stochastic optimization such as prophet inequality, Pandora's box,
stochastic knapsack, stochastic matchings and stochastic submodular
optimization.