Defining Expertise: Applications to Treatment Effect Estimation
Alihan Hüyük Qiyao Wei Alicia Curth Mihaela van der Schaar
Decision-makers are often experts of their domain and take actions based on
their domain knowledge. Doctors, for instance, may prescribe treatments by
predicting the likely outcome of each available treatment. Actions of an expert
thus naturally encode part of their domain knowledge, and can help make
inferences within the same domain: Knowing doctors try to prescribe the best
treatment for their patients, we can tell treatments prescribed more frequently
are likely to be more effective. Yet in machine learning, the fact that most
decision-makers are experts is often overlooked, and "expertise" is seldom
leveraged as an inductive bias. This is especially true for the literature on
treatment effect estimation, where often the only assumption made about actions
is that of overlap. In this paper, we argue that expertise - particularly the
type of expertise the decision-makers of a domain are likely to have - can be
informative in designing and selecting methods for treatment effect estimation.
We formally define two types of expertise, predictive and prognostic, and
demonstrate empirically that: (i) the prominent type of expertise in a domain
significantly influences the performance of different methods in treatment
effect estimation, and (ii) it is possible to predict the type of expertise
present in a dataset, which can provide a quantitative basis for model
selection.