Risk-Aware Reinforcement Learning through Optimal Transport Theory
Ali Baheri
In the dynamic and uncertain environments where reinforcement learning (RL)
operates, risk management becomes a crucial factor in ensuring reliable
decision-making. Traditional RL approaches, while effective in reward
optimization, often overlook the landscape of potential risks. In response,
this paper pioneers the integration of Optimal Transport (OT) theory with RL to
create a risk-aware framework. Our approach modifies the objective function,
ensuring that the resulting policy not only maximizes expected rewards but also
respects risk constraints dictated by OT distances between state visitation
distributions and the desired risk profiles. By leveraging the mathematical
precision of OT, we offer a formulation that elevates risk considerations
alongside conventional RL objectives. Our contributions are substantiated with
a series of theorems, mapping the relationships between risk distributions,
optimal value functions, and policy behaviors. Through the lens of OT, this
work illuminates a promising direction for RL, ensuring a balanced fusion of
reward pursuit and risk awareness.