Continuous Reinforcement Learning-based Dynamic Difficulty Adjustment in
  a Visual Working Memory Game
Masoud Rahimi Hadi Moradi Abdol-hossein Vahabie Hamed Kebriaei
Dynamic Difficulty Adjustment (DDA) is a viable approach to enhance a
player's experience in video games. Recently, Reinforcement Learning (RL)
methods have been employed for DDA in non-competitive games; nevertheless, they
rely solely on discrete state-action space with a small search space. In this
paper, we propose a continuous RL-based DDA methodology for a visual working
memory (VWM) game to handle the complex search space for the difficulty of
memorization. The proposed RL-based DDA tailors game difficulty based on the
player's score and game difficulty in the last trial. We defined a continuous
metric for the difficulty of memorization. Then, we consider the task
difficulty and the vector of difficulty-score as the RL's action and state,
respectively. We evaluated the proposed method through a within-subject
experiment involving 52 subjects. The proposed approach was compared with two
rule-based difficulty adjustment methods in terms of player's score and game
experience measured by a questionnaire. The proposed RL-based approach resulted
in a significantly better game experience in terms of competence, tension, and
negative and positive affect. Players also achieved higher scores and win
rates. Furthermore, the proposed RL-based DDA led to a significantly less
decline in the score in a 20-trial session.