In mathematical statistics, the Kullback–Leibler (KL) divergence (also called relative entropy and I-divergence), denoted D KL ( P ∥ ∥ Q ) , is a type of statistical distance : a measure of how one probability distribution P is different from a second, reference probability distribution Q. Mathematically, it is defined as