What Can AutoML Do For Continual Learning?
Mert Kilickaya Joaquin Vanschoren
This position paper outlines the potential of AutoML for incremental
(continual) learning to encourage more research in this direction. Incremental
learning involves incorporating new data from a stream of tasks and
distributions to learn enhanced deep representations and adapt better to new
tasks. However, a significant limitation of incremental learners is that most
current techniques freeze the backbone architecture, hyperparameters, and the
order & structure of the learning tasks throughout the learning and adaptation
process. We strongly believe that AutoML offers promising solutions to address
these limitations, enabling incremental learning to adapt to more diverse
real-world tasks. Therefore, instead of directly proposing a new method, this
paper takes a step back by posing the question: "What can AutoML do for
incremental learning?" We outline three key areas of research that can
contribute to making incremental learners more dynamic, highlighting concrete
opportunities to apply AutoML methods in novel ways as well as entirely new
challenges for AutoML research.