An explanation that requires the knowledge of a PhD in math or computer science may be suitable for model developers, but may be of little use to a compliance officer, who is responsible for overseeing risk management across a wide swath of bank operations. The level and type of explainability also depends on the model’s use.
In the consumer protection context, consumers’ needs and fairness may define the parameters of the explanation. 12  Castelvevchi, “Can we open,” 20–23. 6  Importantly, consumer protection laws require lenders who decline to offer a consumer credit— or offer credit on materially worse terms than offered to others—to provide the consumer with an explanation of the reasons for the decision.