Towards a Standardized Reinforcement Learning Framework for AAM
  Contingency Management
Luis E. Alvarez Marc W. Brittain Kara Breeden
Advanced Air Mobility (AAM) is the next generation of air transportation that
includes new entrants such as electric vertical takeoff and landing (eVTOL)
aircraft, increasingly autonomous flight operations, and small UAS package
delivery. With these new vehicles and operational concepts comes a desire to
increase densities far beyond what occurs today in and around urban areas, to
utilize new battery technology, and to move toward more autonomously-piloted
aircraft. To achieve these goals, it becomes essential to introduce new safety
management system capabilities that can rapidly assess risk as it evolves
across a span of complex hazards and, if necessary, mitigate risk by executing
appropriate contingencies via supervised or automated decision-making during
flights. Recently, reinforcement learning has shown promise for real-time
decision making across a wide variety of applications including contingency
management. In this work, we formulate the contingency management problem as a
Markov Decision Process (MDP) and integrate the contingency management MDP into
the AAM-Gym simulation framework. This enables rapid prototyping of
reinforcement learning algorithms and evaluation of existing systems, thus
providing a community benchmark for future algorithm development. We report
baseline statistical information for the environment and provide example
performance metrics.