Deep reinforcement learning for machine scheduling: Methodology, the
  state-of-the-art, and future directions
Maziyar Khadivi Todd Charter Marjan Yaghoubi Masoud Jalayer Maryam Ahang Ardeshir Shojaeinasab Homayoun Najjaran
Machine scheduling aims to optimize job assignments to machines while
adhering to manufacturing rules and job specifications. This optimization leads
to reduced operational costs, improved customer demand fulfillment, and
enhanced production efficiency. However, machine scheduling remains a
challenging combinatorial problem due to its NP-hard nature. Deep Reinforcement
Learning (DRL), a key component of artificial general intelligence, has shown
promise in various domains like gaming and robotics. Researchers have explored
applying DRL to machine scheduling problems since 1995. This paper offers a
comprehensive review and comparison of DRL-based approaches, highlighting their
methodology, applications, advantages, and limitations. It categorizes these
approaches based on computational components: conventional neural networks,
encoder-decoder architectures, graph neural networks, and metaheuristic
algorithms. Our review concludes that DRL-based methods outperform exact
solvers, heuristics, and tabular reinforcement learning algorithms in terms of
computation speed and generating near-global optimal solutions. These DRL-based
approaches have been successfully applied to static and dynamic scheduling
across diverse machine environments and job characteristics. However, DRL-based
schedulers face limitations in handling complex operational constraints,
configurable multi-objective optimization, generalization, scalability,
interpretability, and robustness. Addressing these challenges will be a crucial
focus for future research in this field. This paper serves as a valuable
resource for researchers to assess the current state of DRL-based machine
scheduling and identify research gaps. It also aids experts and practitioners
in selecting the appropriate DRL approach for production scheduling.