RL Dreams: Policy Gradient Optimization for Score Distillation based 3D
  Generation
Aradhya N. Mathur Phu Pham Aniket Bera Ojaswa Sharma
3D generation has rapidly accelerated in the past decade owing to the
progress in the field of generative modeling. Score Distillation Sampling (SDS)
based rendering has improved 3D asset generation to a great extent. Further,
the recent work of Denoising Diffusion Policy Optimization (DDPO) demonstrates
that the diffusion process is compatible with policy gradient methods and has
been demonstrated to improve the 2D diffusion models using an aesthetic scoring
function. We first show that this aesthetic scorer acts as a strong guide for a
variety of SDS-based methods and demonstrates its effectiveness in text-to-3D
synthesis. Further, we leverage the DDPO approach to improve the quality of the
3D rendering obtained from 2D diffusion models. Our approach, DDPO3D, employs
the policy gradient method in tandem with aesthetic scoring. To the best of our
knowledge, this is the first method that extends policy gradient methods to 3D
score-based rendering and shows improvement across SDS-based methods such as
DreamGaussian, which are currently driving research in text-to-3D synthesis.
Our approach is compatible with score distillation-based methods, which would
facilitate the integration of diverse reward functions into the generative
process. Our project page can be accessed via https://ddpo3d.github.io.