Optimal Sequential Decision-Making in Geosteering: A Reinforcement
  Learning Approach
Ressi Bonti Muhammad Sergey Alyaev Reidar Brumer Bratvold
Trajectory adjustment decisions throughout the drilling process, called
geosteering, affect subsequent choices and information gathering, thus
resulting in a coupled sequential decision problem. Previous works on applying
decision optimization methods in geosteering rely on greedy optimization or
Approximate Dynamic Programming (ADP). Either decision optimization method
requires explicit uncertainty and objective function models, making developing
decision optimization methods for complex and realistic geosteering
environments challenging to impossible. We use the Deep Q-Network (DQN) method,
a model-free reinforcement learning (RL) method that learns directly from the
decision environment, to optimize geosteering decisions. The expensive
computations for RL are handled during the offline training stage. Evaluating
DQN needed for real-time decision support takes milliseconds and is faster than
the traditional alternatives. Moreover, for two previously published synthetic
geosteering scenarios, our results show that RL achieves high-quality outcomes
comparable to the quasi-optimal ADP. Yet, the model-free nature of RL means
that by replacing the training environment, we can extend it to problems where
the solution to ADP is prohibitively expensive to compute. This flexibility
will allow applying it to more complex environments and make hybrid versions
trained with real data in the future.