Most RL systems assume that the rest-of-the-world (the thing being controlled by the RL system) can be modeled as a Markov Decision Process (MDP) or even a Partially Observable Markov Decision Process (POMDP).