DeepCalliFont: Few-shot Chinese Calligraphy Font Synthesis by
  Integrating Dual-modality Generative Models
Yitian Liu Zhouhui Lian
Few-shot font generation, especially for Chinese calligraphy fonts, is a
challenging and ongoing problem. With the help of prior knowledge that is
mainly based on glyph consistency assumptions, some recently proposed methods
can synthesize high-quality Chinese glyph images. However, glyphs in
calligraphy font styles often do not meet these assumptions. To address this
problem, we propose a novel model, DeepCalliFont, for few-shot Chinese
calligraphy font synthesis by integrating dual-modality generative models.
Specifically, the proposed model consists of image synthesis and sequence
generation branches, generating consistent results via a dual-modality
representation learning strategy. The two modalities (i.e., glyph images and
writing sequences) are properly integrated using a feature recombination module
and a rasterization loss function. Furthermore, a new pre-training strategy is
adopted to improve the performance by exploiting large amounts of uni-modality
data. Both qualitative and quantitative experiments have been conducted to
demonstrate the superiority of our method to other state-of-the-art approaches
in the task of few-shot Chinese calligraphy font synthesis. The source code can
be found at https://github.com/lsflyt-pku/DeepCalliFont.