HyperTime: Hyperparameter Optimization for Combating Temporal
  Distribution Shifts
Shaokun Zhang Yiran Wu Zhonghua Zheng Qingyun Wu Chi Wang
In this work, we propose a hyperparameter optimization method named
\emph{HyperTime} to find hyperparameters robust to potential temporal
distribution shifts in the unseen test data. Our work is motivated by an
important observation that it is, in many cases, possible to achieve temporally
robust predictive performance via hyperparameter optimization. Based on this
observation, we leverage the `worst-case-oriented' philosophy from the robust
optimization literature to help find such robust hyperparameter configurations.
HyperTime imposes a lexicographic priority order on average validation loss and
worst-case validation loss over chronological validation sets. We perform a
theoretical analysis on the upper bound of the expected test loss, which
reveals the unique advantages of our approach. We also demonstrate the strong
empirical performance of the proposed method on multiple machine learning tasks
with temporal distribution shifts.