Decision-Dependent Distributionally Robust Markov Decision Process
  Method in Dynamic Epidemic Control
Jun Song William Yang Chaoyue Zhao
In this paper, we present a Distributionally Robust Markov Decision Process
(DRMDP) approach for addressing the dynamic epidemic control problem. The
Susceptible-Exposed-Infectious-Recovered (SEIR) model is widely used to
represent the stochastic spread of infectious diseases, such as COVID-19. While
Markov Decision Processes (MDP) offers a mathematical framework for identifying
optimal actions, such as vaccination and transmission-reducing intervention, to
combat disease spreading according to the SEIR model. However, uncertainties in
these scenarios demand a more robust approach that is less reliant on
error-prone assumptions. The primary objective of our study is to introduce a
new DRMDP framework that allows for an ambiguous distribution of transition
dynamics. Specifically, we consider the worst-case distribution of these
transition probabilities within a decision-dependent ambiguity set. To overcome
the computational complexities associated with policy determination, we propose
an efficient Real-Time Dynamic Programming (RTDP) algorithm that is capable of
computing optimal policies based on the reformulated DRMDP model in an
accurate, timely, and scalable manner. Comparative analysis against the classic
MDP model demonstrates that the DRMDP achieves a lower proportion of infections
and susceptibilities at a reduced cost.