Temporal Transfer Learning for Traffic Optimization with Coarse-grained
  Advisory Autonomy
Jung-Hoon Cho Sirui Li Jeongyun Kim Cathy Wu
The recent development of connected and automated vehicle (CAV) technologies
has spurred investigations to optimize dense urban traffic. This paper
considers advisory autonomy, in which real-time driving advisories are issued
to drivers, thus blending the CAV and the human driver. Due to the complexity
of traffic systems, recent studies of coordinating CAVs have resorted to
leveraging deep reinforcement learning (RL). Advisory autonomy is formalized as
zero-order holds, and we consider a range of hold duration from 0.1 to 40
seconds. However, despite the similarity of the higher frequency tasks on CAVs,
a direct application of deep RL fails to be generalized to advisory autonomy
tasks. We introduce Temporal Transfer Learning (TTL) algorithms to select
source tasks, systematically leveraging the temporal structure to solve the
full range of tasks. TTL selects the most suitable source tasks to maximize the
performance of the range of tasks. We validate our algorithms on diverse
mixed-traffic scenarios, demonstrating that TTL more reliably solves the tasks
than baselines. This paper underscores the potential of coarse-grained advisory
autonomy with TTL in traffic flow optimization.