Filtered-Guided Diffusion: Fast Filter Guidance for Black-Box Diffusion
  Models
Zeqi Gu Abe Davis
Recent advances in diffusion-based generative models have shown incredible
promise for Image-to-Image translation and editing. Most recent work in this
space relies on additional training or architecture-specific adjustments to the
diffusion process. In this work, we show that much of this low-level control
can be achieved without additional training or any access to features of the
diffusion model. Our method simply applies a filter to the input of each
diffusion step based on the output of the previous step in an adaptive manner.
Notably, this approach does not depend on any specific architecture or sampler
and can be done without access to internal features of the network, making it
easy to combine with other techniques, samplers, and diffusion architectures.
Furthermore, it has negligible cost to performance, and allows for more
continuous adjustment of guidance strength than other approaches. We show FGD
offers a fast and strong baseline that is competitive with recent
architecture-dependent approaches. Furthermore, FGD can also be used as a
simple add-on to enhance the structural guidance of other state-of-the-art I2I
methods. Finally, our derivation of this method helps to understand the impact
of self attention, a key component of other recent architecture-specific I2I
approaches, in a more architecture-independent way. Project page:
https://github.com/jaclyngu/FilteredGuidedDiffusion