PaddingFlow: Improving Normalizing Flows with Padding-Dimensional Noise
Qinglong Meng Chongkun Xia Xueqian Wang
Normalizing flow is a generative modeling approach with efficient sampling.
However, Flow-based models suffer two issues, which are manifold and discrete
data. If the target distribution is a manifold, which means the dimension of
the latent target distribution and the dimension of the data distribution are
unmatched, flow-based models might perform badly. Discrete data makes
flow-based models collapse into a degenerate mixture of point masses. In this
paper, to sidestep such two issues we propose PaddingFlow, a novel
dequantization method, which improves normalizing flows with
padding-dimensional noise. PaddingFlow is easy to implement, computationally
cheap, widely suitable for various tasks, and generates samples that are
unbiased estimations of the data. Especially, our method can overcome the
limitation of existing dequantization methods that have to change the data
distribution, which might degrade performance. We validate our method on the
main benchmarks of unconditional density estimation, including five tabular
datasets and four image datasets for VAE models, and the IK experiments which
are conditional density estimation. The results show that PaddingFlow can
provide improvement on all tasks in this paper.