Expressed in the language of Bayesian inference, D KL ( P ∥ ∥ Q ) is a measure of the information gained by revising one's beliefs from the prior probability distribution Q to the posterior probability distribution P. In other words, it is the amount of information lost when Q is used to approximate P.