CogCartoon: Towards Practical Story Visualization
Zhongyang Zhu Jie Tang
The state-of-the-art methods for story visualization demonstrate a
significant demand for training data and storage, as well as limited
flexibility in story presentation, thereby rendering them impractical for
real-world applications. We introduce CogCartoon, a practical story
visualization method based on pre-trained diffusion models. To alleviate
dependence on data and storage, we propose an innovative strategy of
character-plugin generation that can represent a specific character as a
compact 316 KB plugin by using a few training samples. To facilitate enhanced
flexibility, we employ a strategy of plugin-guided and layout-guided inference,
enabling users to seamlessly incorporate new characters and custom layouts into
the generated image results at their convenience. We have conducted
comprehensive qualitative and quantitative studies, providing compelling
evidence for the superiority of CogCartoon over existing methodologies.
Moreover, CogCartoon demonstrates its power in tackling challenging tasks,
including long story visualization and realistic style story visualization.