 It quantifies the redundancy or dependency among a set of "n" random variables.For a given set of "n" random variables formula_1, the total correlation formula_2 is defined as the Kullback–Leibler divergence from the joint distribution formula_3 to the independent distribution of formula_4,

This divergence reduces to the simpler difference of entropies,
where formula_7 is the information entropy of variable formula_8, and formula_9 is the joint entropy of the variable set formula_1.In terms of the discrete probability distributions on variables formula_11, the total correlation is given by

The total correlation is the amount of information "shared" among the variables in the set.