Diffutoon: High-Resolution Editable Toon Shading via Diffusion Models
Zhongjie Duan Chengyu Wang Cen Chen Weining Qian Jun Huang
Toon shading is a type of non-photorealistic rendering task of animation. Its
primary purpose is to render objects with a flat and stylized appearance. As
diffusion models have ascended to the forefront of image synthesis
methodologies, this paper delves into an innovative form of toon shading based
on diffusion models, aiming to directly render photorealistic videos into anime
styles. In video stylization, extant methods encounter persistent challenges,
notably in maintaining consistency and achieving high visual quality. In this
paper, we model the toon shading problem as four subproblems: stylization,
consistency enhancement, structure guidance, and colorization. To address the
challenges in video stylization, we propose an effective toon shading approach
called \textit{Diffutoon}. Diffutoon is capable of rendering remarkably
detailed, high-resolution, and extended-duration videos in anime style. It can
also edit the content according to prompts via an additional branch. The
efficacy of Diffutoon is evaluated through quantitive metrics and human
evaluation. Notably, Diffutoon surpasses both open-source and closed-source
baseline approaches in our experiments. Our work is accompanied by the release
of both the source code and example videos on Github (Project page:
https://ecnu-cilab.github.io/DiffutoonProjectPage/).