Consistent Mesh Diffusion
Julian Knodt Xifeng Gao
Given a 3D mesh with a UV parameterization, we introduce a novel approach to
generating textures from text prompts. While prior work uses optimization from
Text-to-Image Diffusion models to generate textures and geometry, this is slow
and requires significant compute resources. Alternatively, there are projection
based approaches that use the same Text-to-Image models that paint images onto
a mesh, but lack consistency at different viewing angles, we propose a method
that uses a single Depth-to-Image diffusion network, and generates a single
consistent texture when rendered on the 3D surface by first unifying multiple
2D image's diffusion paths, and hoisting that to 3D with
MultiDiffusion~\cite{multidiffusion}. We demonstrate our approach on a dataset
containing 30 meshes, taking approximately 5 minutes per mesh. To evaluate the
quality of our approach, we use CLIP-score~\cite{clipscore} and Frechet
Inception Distance (FID)~\cite{frechet} to evaluate the quality of the
rendering, and show our improvement over prior work.