On the Efficient Marginalization of Probabilistic Sequence Models
Alex Boyd
Real-world data often exhibits sequential dependence, across diverse domains
such as human behavior, medicine, finance, and climate modeling. Probabilistic
methods capture the inherent uncertainty associated with prediction in these
contexts, with autoregressive models being especially prominent. This
dissertation focuses on using autoregressive models to answer complex
probabilistic queries that go beyond single-step prediction, such as the timing
of future events or the likelihood of a specific event occurring before
another. In particular, we develop a broad class of novel and efficient
approximation techniques for marginalization in sequential models that are
model-agnostic. These techniques rely solely on access to and sampling from
next-step conditional distributions of a pre-trained autoregressive model,
including both traditional parametric models as well as more recent neural
autoregressive models. Specific approaches are presented for discrete
sequential models, for marked temporal point processes, and for stochastic jump
processes, each tailored to a well-defined class of informative, long-range
probabilistic queries.