Full Bayesian Significance Testing for Neural Networks
Zehua Liu Zimeng Li Jingyuan Wang Yue He
Significance testing aims to determine whether a proposition about the
population distribution is the truth or not given observations. However,
traditional significance testing often needs to derive the distribution of the
testing statistic, failing to deal with complex nonlinear relationships. In
this paper, we propose to conduct Full Bayesian Significance Testing for neural
networks, called \textit{n}FBST, to overcome the limitation in relationship
characterization of traditional approaches. A Bayesian neural network is
utilized to fit the nonlinear and multi-dimensional relationships with small
errors and avoid hard theoretical derivation by computing the evidence value.
Besides, \textit{n}FBST can test not only global significance but also local
and instance-wise significance, which previous testing methods don't focus on.
Moreover, \textit{n}FBST is a general framework that can be extended based on
the measures selected, such as Grad-\textit{n}FBST, LRP-\textit{n}FBST,
DeepLIFT-\textit{n}FBST, LIME-\textit{n}FBST. A range of experiments on both
simulated and real data are conducted to show the advantages of our method.