Alchemist: Parametric Control of Material Properties with Diffusion
  Models
Prafull Sharma Varun Jampani Yuanzhen Li Xuhui Jia Dmitry Lagun Fredo Durand William T. Freeman Mark Matthews
We propose a method to control material attributes of objects like roughness,
metallic, albedo, and transparency in real images. Our method capitalizes on
the generative prior of text-to-image models known for photorealism, employing
a scalar value and instructions to alter low-level material properties.
Addressing the lack of datasets with controlled material attributes, we
generated an object-centric synthetic dataset with physically-based materials.
Fine-tuning a modified pre-trained text-to-image model on this synthetic
dataset enables us to edit material properties in real-world images while
preserving all other attributes. We show the potential application of our model
to material edited NeRFs.