Aikman et al (2014) discuss where the classification of failure departs from Laeven and Valencia (2010). 23  All speeches are available online at www.bankofengland.co.uk/speeches  23  instrument if policymakers have a low tolerance for failure. That matters if, for example, the costs of higher capital requirements increase non-linearly (Greenwood et al (2017)). These points are also evident if we assess individually the performance of the RWCR and NSFR. Chart 18 and Table 3 below show that hit rates of 80 or 90% can only be achieved with high false alarm rates and stringent calibrations of these metrics. Overall, each metric individually does somewhat worse than the leverage ratio in balancing hit and false alarm rates. And similar results hold when the loan to deposit (LTD) ratio is considered instead of the NSFR in the wider sample (Chart 19).
We define the ‘hit rate’ as the number of banks which had a leverage ratio below 3% at the end of 2006 and subsequently failed during the crisis, relative to the total number of banks that failed. And we define the ‘false alarm rate’ as the number of banks with a leverage ratio below 3% which survived, relative to the total number of banks that survived. If a 3% leverage ratio could perfectly discriminate, its hit rate would be 100% and false alarm rate 0%. Now suppose that we have flexibility over the cut-off threshold necessary to achieve a particular hit rate, x. At the same time, we wish to minimise false alarms. As the leverage ratio cut-off increases, the hit rate and false alarm rate must both go up. The key question is by how much each goes up – the relative balance of marginal benefits and marginal costs of hits and false alarms – as the leverage ratio cut-off increases. Charts 17 and 18 plot this for the 76-bank sample. Chart 17 plots the settings of the leverage ratio needed to achieve particular target hit rates. Chart 18, meanwhile, plots what is referred to as the ‘receiver operating characteristic’ (ROC) curve. Using the sequence of cut-off thresholds for the leverage ratio from Chart 17, this plots the sequence of associated hit rates and corresponding false alarm rates at different settings of the leverage ratio, alongside the 45 degree line which corresponds to the performance of a completely uninformative metric.