Discrete Distribution Networks
Lei Yang
We introduce a novel generative model, the Discrete Distribution Networks
(DDN), that approximates data distribution using hierarchical discrete
distributions. We posit that since the features within a network inherently
contain distributional information, liberating the network from a single output
to concurrently generate multiple samples proves to be highly effective.
Therefore, DDN fits the target distribution, including continuous ones, by
generating multiple discrete sample points. To capture finer details of the
target data, DDN selects the output that is closest to the Ground Truth (GT)
from the coarse results generated in the first layer. This selected output is
then fed back into the network as a condition for the second layer, thereby
generating new outputs more similar to the GT. As the number of DDN layers
increases, the representational space of the outputs expands exponentially, and
the generated samples become increasingly similar to the GT. This hierarchical
output pattern of discrete distributions endows DDN with two intriguing
properties: highly compressed representation and more general zero-shot
conditional generation. We demonstrate the efficacy of DDN and these intriguing
properties through experiments on CIFAR-10 and FFHQ.