DCT: Dual Channel Training of Action Embeddings for Reinforcement
  Learning with Large Discrete Action Spaces
Pranavi Pathakota Hardik Meisheri Harshad Khadilkar
The ability to learn robust policies while generalizing over large discrete
action spaces is an open challenge for intelligent systems, especially in noisy
environments that face the curse of dimensionality. In this paper, we present a
novel framework to efficiently learn action embeddings that simultaneously
allow us to reconstruct the original action as well as to predict the expected
future state. We describe an encoder-decoder architecture for action embeddings
with a dual channel loss that balances between action reconstruction and state
prediction accuracy. We use the trained decoder in conjunction with a standard
reinforcement learning algorithm that produces actions in the embedding space.
Our architecture is able to outperform two competitive baselines in two diverse
environments: a 2D maze environment with more than 4000 discrete noisy actions,
and a product recommendation task that uses real-world e-commerce transaction
data. Empirical results show that the model results in cleaner action
embeddings, and the improved representations help learn better policies with
earlier convergence.