Machine Learning and Knowledge: Why Robustness Matters
Jonathan Vandenburgh
Trusting machine learning algorithms requires having confidence in their
outputs. Confidence is typically interpreted in terms of model reliability,
where a model is reliable if it produces a high proportion of correct outputs.
However, model reliability does not address concerns about the robustness of
machine learning models, such as models relying on the wrong features or
variations in performance based on context. I argue that the epistemic
dimension of trust can instead be understood through the concept of knowledge,
where the trustworthiness of an algorithm depends on whether its users are in
the position to know that its outputs are correct. Knowledge requires beliefs
to be formed for the right reasons and to be robust to error, so machine
learning algorithms can only provide knowledge if they work well across
counterfactual scenarios and if they make decisions based on the right
features. This, I argue, can explain why we should care about model properties
like interpretability, causal shortcut independence, and distribution shift
robustness even if such properties are not required for model reliability.