StylizedGS: Controllable Stylization for 3D Gaussian Splatting
Dingxi Zhang Zhuoxun Chen Yu-Jie Yuan Fang-Lue Zhang Zhenliang He Shiguang Shan Lin Gao
With the rapid development of XR, 3D generation and editing are becoming more
and more important, among which, stylization is an important tool of 3D
appearance editing. It can achieve consistent 3D artistic stylization given a
single reference style image and thus is a user-friendly editing way. However,
recent NeRF-based 3D stylization methods face efficiency issues that affect the
actual user experience and the implicit nature limits its ability to transfer
the geometric pattern styles. Additionally, the ability for artists to exert
flexible control over stylized scenes is considered highly desirable, fostering
an environment conducive to creative exploration. In this paper, we introduce
StylizedGS, a 3D neural style transfer framework with adaptable control over
perceptual factors based on 3D Gaussian Splatting (3DGS) representation. The
3DGS brings the benefits of high efficiency. We propose a GS filter to
eliminate floaters in the reconstruction which affects the stylization effects
before stylization. Then the nearest neighbor-based style loss is introduced to
achieve stylization by fine-tuning the geometry and color parameters of 3DGS,
while a depth preservation loss with other regularizations is proposed to
prevent the tampering of geometry content. Moreover, facilitated by specially
designed losses, StylizedGS enables users to control color, stylized scale and
regions during the stylization to possess customized capabilities. Our method
can attain high-quality stylization results characterized by faithful
brushstrokes and geometric consistency with flexible controls. Extensive
experiments across various scenes and styles demonstrate the effectiveness and
efficiency of our method concerning both stylization quality and inference FPS.