Stage-Aware Learning for Dynamic Treatments
Hanwen Ye Wenzhuo Zhou Ruoqing Zhu Annie Qu
Recent advances in dynamic treatment regimes (DTRs) provide powerful optimal
treatment searching algorithms, which are tailored to individuals' specific
needs and able to maximize their expected clinical benefits. However, existing
algorithms could suffer from insufficient sample size under optimal treatments,
especially for chronic diseases involving long stages of decision-making. To
address these challenges, we propose a novel individualized learning method
which estimates the DTR with a focus on prioritizing alignment between the
observed treatment trajectory and the one obtained by the optimal regime across
decision stages. By relaxing the restriction that the observed trajectory must
be fully aligned with the optimal treatments, our approach substantially
improves the sample efficiency and stability of inverse probability weighted
based methods. In particular, the proposed learning scheme builds a more
general framework which includes the popular outcome weighted learning
framework as a special case of ours. Moreover, we introduce the notion of stage
importance scores along with an attention mechanism to explicitly account for
heterogeneity among decision stages. We establish the theoretical properties of
the proposed approach, including the Fisher consistency and finite-sample
performance bound. Empirically, we evaluate the proposed method in extensive
simulated environments and a real case study for COVID-19 pandemic.