Detecting and Learning Out-of-Distribution Data in the Open world:
  Algorithm and Theory
Yiyou Sun
This thesis makes considerable contributions to the realm of machine
learning, specifically in the context of open-world scenarios where systems
face previously unseen data and contexts. Traditional machine learning models
are usually trained and tested within a fixed and known set of classes, a
condition known as the closed-world setting. While this assumption works in
controlled environments, it falls short in real-world applications where new
classes or categories of data can emerge dynamically and unexpectedly. To
address this, our research investigates two intertwined steps essential for
open-world machine learning: Out-of-distribution (OOD) Detection and Open-world
Representation Learning (ORL). OOD detection focuses on identifying instances
from unknown classes that fall outside the model's training distribution. This
process reduces the risk of making overly confident, erroneous predictions
about unfamiliar inputs. Moving beyond OOD detection, ORL extends the
capabilities of the model to not only detect unknown instances but also learn
from and incorporate knowledge about these new classes. By delving into these
research problems of open-world learning, this thesis contributes both
algorithmic solutions and theoretical foundations, which pave the way for
building machine learning models that are not only performant but also reliable
in the face of the evolving complexities of the real world.