Multi-Agent Learning of Efficient Fulfilment and Routing Strategies in
  E-Commerce
Omkar Shelke Pranavi Pathakota Anandsingh Chauhan Harshad Khadilkar Hardik Meisheri Balaraman Ravindran
This paper presents an integrated algorithmic framework for minimising
product delivery costs in e-commerce (known as the cost-to-serve or C2S). One
of the major challenges in e-commerce is the large volume of spatio-temporally
diverse orders from multiple customers, each of which has to be fulfilled from
one of several warehouses using a fleet of vehicles. This results in two levels
of decision-making: (i) selection of a fulfillment node for each order
(including the option of deferral to a future time), and then (ii) routing of
vehicles (each of which can carry multiple orders originating from the same
warehouse). We propose an approach that combines graph neural networks and
reinforcement learning to train the node selection and vehicle routing agents.
We include real-world constraints such as warehouse inventory capacity, vehicle
characteristics such as travel times, service times, carrying capacity, and
customer constraints including time windows for delivery. The complexity of
this problem arises from the fact that outcomes (rewards) are driven both by
the fulfillment node mapping as well as the routing algorithms, and are
spatio-temporally distributed. Our experiments show that this algorithmic
pipeline outperforms pure heuristic policies.