Hierarchical Partitioning Forecaster
Christopher Mattern
In this work we consider a new family of algorithms for sequential
prediction, Hierarchical Partitioning Forecasters (HPFs). Our goal is to
provide appealing theoretical - regret guarantees on a powerful model class -
and practical - empirical performance comparable to deep networks - properties
at the same time. We built upon three principles: hierarchically partitioning
the feature space into sub-spaces, blending forecasters specialized to each
sub-space and learning HPFs via local online learning applied to these
individual forecasters. Following these principles allows us to obtain regret
guarantees, where Constant Partitioning Forecasters (CPFs) serve as competitor.
A CPF partitions the feature space into sub-spaces and predicts with a fixed
forecaster per sub-space. Fixing a hierarchical partition $\mathcal H$ and
considering any CPF with a partition that can be constructed using elements of
$\mathcal H$ we provide two guarantees: first, a generic one that unveils how
local online learning determines regret of learning the entire HPF online;
second, a concrete instance that considers HPF with linear forecasters (LHPF)
and exp-concave losses where we obtain $O(k \log T)$ regret for sequences of
length $T$ where $k$ is a measure of complexity for the competing CPF. Finally,
we provide experiments that compare LHPF to various baselines, including state
of the art deep learning models, in precipitation nowcasting. Our results
indicate that LHPF is competitive in various settings.