3DTextureTransformer: Geometry Aware Texture Generation for Arbitrary
  Mesh Topology
Dharma KC Clayton T. Morrison
Learning to generate textures for a novel 3D mesh given a collection of 3D
meshes and real-world 2D images is an important problem with applications in
various domains such as 3D simulation, augmented and virtual reality, gaming,
architecture, and design. Existing solutions either do not produce high-quality
textures or deform the original high-resolution input mesh topology into a
regular grid to make this generation easier but also lose the original mesh
topology. In this paper, we present a novel framework called the
3DTextureTransformer that enables us to generate high-quality textures without
deforming the original, high-resolution input mesh. Our solution, a hybrid of
geometric deep learning and StyleGAN-like architecture, is flexible enough to
work on arbitrary mesh topologies and also easily extensible to texture
generation for point cloud representations. Our solution employs a
message-passing framework in 3D in conjunction with a StyleGAN-like
architecture for 3D texture generation. The architecture achieves
state-of-the-art performance among a class of solutions that can learn from a
collection of 3D geometry and real-world 2D images while working with any
arbitrary mesh topology.