A Systems Theoretic Approach to Online Machine Learning
Anli du Preez Peter A. Beling Tyler Cody
The machine learning formulation of online learning is incomplete from a
systems theoretic perspective. Typically, machine learning research emphasizes
domains and tasks, and a problem solving worldview. It focuses on algorithm
parameters, features, and samples, and neglects the perspective offered by
considering system structure and system behavior or dynamics. Online learning
is an active field of research and has been widely explored in terms of
statistical theory and computational algorithms, however, in general, the
literature still lacks formal system theoretical frameworks for modeling online
learning systems and resolving systems-related concept drift issues.
Furthermore, while the machine learning formulation serves to classify methods
and literature, the systems theoretic formulation presented herein serves to
provide a framework for the top-down design of online learning systems,
including a novel definition of online learning and the identification of key
design parameters. The framework is formulated in terms of input-output systems
and is further divided into system structure and system behavior. Concept drift
is a critical challenge faced in online learning, and this work formally
approaches it as part of the system behavior characteristics. Healthcare
provider fraud detection using machine learning is used as a case study
throughout the paper to ground the discussion in a real-world online learning
challenge.