Evaluating Dynamic Conditional Quantile Treatment Effects with
  Applications in Ridesharing
Ting Li Chengchun Shi Zhaohua Lu Yi Li Hongtu Zhu
Many modern tech companies, such as Google, Uber, and Didi, utilize online
experiments (also known as A/B testing) to evaluate new policies against
existing ones. While most studies concentrate on average treatment effects,
situations with skewed and heavy-tailed outcome distributions may benefit from
alternative criteria, such as quantiles. However, assessing dynamic quantile
treatment effects (QTE) remains a challenge, particularly when dealing with
data from ride-sourcing platforms that involve sequential decision-making
across time and space. In this paper, we establish a formal framework to
calculate QTE conditional on characteristics independent of the treatment.
Under specific model assumptions, we demonstrate that the dynamic conditional
QTE (CQTE) equals the sum of individual CQTEs across time, even though the
conditional quantile of cumulative rewards may not necessarily equate to the
sum of conditional quantiles of individual rewards. This crucial insight
significantly streamlines the estimation and inference processes for our target
causal estimand. We then introduce two varying coefficient decision process
(VCDP) models and devise an innovative method to test the dynamic CQTE.
Moreover, we expand our approach to accommodate data from spatiotemporal
dependent experiments and examine both conditional quantile direct and indirect
effects. To showcase the practical utility of our method, we apply it to three
real-world datasets from a ride-sourcing platform. Theoretical findings and
comprehensive simulation studies further substantiate our proposal.