Diffusion Model for Data-Driven Black-Box Optimization
Zihao Li Hui Yuan Kaixuan Huang Chengzhuo Ni Yinyu Ye Minshuo Chen Mengdi Wang
Generative AI has redefined artificial intelligence, enabling the creation of
innovative content and customized solutions that drive business practices into
a new era of efficiency and creativity. In this paper, we focus on diffusion
models, a powerful generative AI technology, and investigate their potential
for black-box optimization over complex structured variables. Consider the
practical scenario where one wants to optimize some structured design in a
high-dimensional space, based on massive unlabeled data (representing design
variables) and a small labeled dataset. We study two practical types of labels:
1) noisy measurements of a real-valued reward function and 2) human preference
based on pairwise comparisons. The goal is to generate new designs that are
near-optimal and preserve the designed latent structures. Our proposed method
reformulates the design optimization problem into a conditional sampling
problem, which allows us to leverage the power of diffusion models for modeling
complex distributions. In particular, we propose a reward-directed conditional
diffusion model, to be trained on the mixed data, for sampling a near-optimal
solution conditioned on high predicted rewards. Theoretically, we establish
sub-optimality error bounds for the generated designs. The sub-optimality gap
nearly matches the optimal guarantee in off-policy bandits, demonstrating the
efficiency of reward-directed diffusion models for black-box optimization.
Moreover, when the data admits a low-dimensional latent subspace structure, our
model efficiently generates high-fidelity designs that closely respect the
latent structure. We provide empirical experiments validating our model in
decision-making and content-creation tasks.