Inference for relative sparsity
Samuel J. Weisenthal Sally W. Thurston Ashkan Ertefaie
In healthcare, there is much interest in estimating policies, or mappings
from covariates to treatment decisions. Recently, there is also interest in
constraining these estimated policies to the standard of care, which generated
the observed data. A relative sparsity penalty was proposed to derive policies
that have sparse, explainable differences from the standard of care,
facilitating justification of the new policy. However, the developers of this
penalty only considered estimation, not inference. Here, we develop inference
for the relative sparsity objective function, because characterizing
uncertainty is crucial to applications in medicine. Further, in the relative
sparsity work, the authors only considered the single-stage decision case;
here, we consider the more general, multi-stage case. Inference is difficult,
because the relative sparsity objective depends on the unpenalized value
function, which is unstable and has infinite estimands in the binary action
case. Further, one must deal with a non-differentiable penalty. To tackle these
issues, we nest a weighted Trust Region Policy Optimization function within a
relative sparsity objective, implement an adaptive relative sparsity penalty,
and propose a sample-splitting framework for post-selection inference. We study
the asymptotic behavior of our proposed approaches, perform extensive
simulations, and analyze a real, electronic health record dataset.