Worth of knowledge in deep learning
Hao Xu Yuntian Chen Dongxiao Zhang
Knowledge constitutes the accumulated understanding and experience that
humans use to gain insight into the world. In deep learning, prior knowledge is
essential for mitigating shortcomings of data-driven models, such as data
dependence, generalization ability, and compliance with constraints. To enable
efficient evaluation of the worth of knowledge, we present a framework inspired
by interpretable machine learning. Through quantitative experiments, we assess
the influence of data volume and estimation range on the worth of knowledge.
Our findings elucidate the complex relationship between data and knowledge,
including dependence, synergistic, and substitution effects. Our model-agnostic
framework can be applied to a variety of common network architectures,
providing a comprehensive understanding of the role of prior knowledge in deep
learning models. It can also be used to improve the performance of informed
machine learning, as well as distinguish improper prior knowledge.