Markov Decision Process with an External Temporal Process
Ranga Shaarad Ayyagari Ambedkar Dukkipati
Most reinforcement learning algorithms treat the context under which they
operate as a stationary, isolated and undisturbed environment. However, in the
real world, the environment is constantly changing due to a variety of external
influences. To address this problem, we study Markov Decision Processes (MDP)
under the influence of an external temporal process. We formalize this notion
and discuss conditions under which the problem becomes tractable with suitable
solutions. We propose a policy iteration algorithm to solve this problem and
theoretically analyze its performance.