DRL4Route: A Deep Reinforcement Learning Framework for Pick-up and
  Delivery Route Prediction
Xiaowei Mao Haomin Wen Hengrui Zhang Huaiyu Wan Lixia Wu Jianbin Zheng Haoyuan Hu Youfang Lin
Pick-up and Delivery Route Prediction (PDRP), which aims to estimate the
future service route of a worker given his current task pool, has received
rising attention in recent years. Deep neural networks based on supervised
learning have emerged as the dominant model for the task because of their
powerful ability to capture workers' behavior patterns from massive historical
data. Though promising, they fail to introduce the non-differentiable test
criteria into the training process, leading to a mismatch in training and test
criteria. Which considerably trims down their performance when applied in
practical systems. To tackle the above issue, we present the first attempt to
generalize Reinforcement Learning (RL) to the route prediction task, leading to
a novel RL-based framework called DRL4Route. It combines the behavior-learning
abilities of previous deep learning models with the non-differentiable
objective optimization ability of reinforcement learning. DRL4Route can serve
as a plug-and-play component to boost the existing deep learning models. Based
on the framework, we further implement a model named DRL4Route-GAE for PDRP in
logistic service. It follows the actor-critic architecture which is equipped
with a Generalized Advantage Estimator that can balance the bias and variance
of the policy gradient estimates, thus achieving a more optimal policy.
Extensive offline experiments and the online deployment show that DRL4Route-GAE
improves Location Square Deviation (LSD) by 0.9%-2.7%, and Accuracy@3 (ACC@3)
by 2.4%-3.2% over existing methods on the real-world dataset.