Learning Computational Efficient Bots with Costly Features
Anthony Kobanda Valliappan C. A. Joshua Romoff Ludovic Denoyer
Deep reinforcement learning (DRL) techniques have become increasingly used in
various fields for decision-making processes. However, a challenge that often
arises is the trade-off between both the computational efficiency of the
decision-making process and the ability of the learned agent to solve a
particular task. This is particularly critical in real-time settings such as
video games where the agent needs to take relevant decisions at a very high
frequency, with a very limited inference time.
  In this work, we propose a generic offline learning approach where the
computation cost of the input features is taken into account. We derive the
Budgeted Decision Transformer as an extension of the Decision Transformer that
incorporates cost constraints to limit its cost at inference. As a result, the
model can dynamically choose the best input features at each timestep. We
demonstrate the effectiveness of our method on several tasks, including D4RL
benchmarks and complex 3D environments similar to those found in video games,
and show that it can achieve similar performance while using significantly
fewer computational resources compared to classical approaches.