Spec-Gaussian: Anisotropic View-Dependent Appearance for 3D Gaussian
  Splatting
Ziyi Yang Xinyu Gao Yangtian Sun Yihua Huang Xiaoyang Lyu Wen Zhou Shaohui Jiao Xiaojuan Qi Xiaogang Jin
The recent advancements in 3D Gaussian splatting (3D-GS) have not only
facilitated real-time rendering through modern GPU rasterization pipelines but
have also attained state-of-the-art rendering quality. Nevertheless, despite
its exceptional rendering quality and performance on standard datasets, 3D-GS
frequently encounters difficulties in accurately modeling specular and
anisotropic components. This issue stems from the limited ability of spherical
harmonics (SH) to represent high-frequency information. To overcome this
challenge, we introduce Spec-Gaussian, an approach that utilizes an anisotropic
spherical Gaussian (ASG) appearance field instead of SH for modeling the
view-dependent appearance of each 3D Gaussian. Additionally, we have developed
a coarse-to-fine training strategy to improve learning efficiency and eliminate
floaters caused by overfitting in real-world scenes. Our experimental results
demonstrate that our method surpasses existing approaches in terms of rendering
quality. Thanks to ASG, we have significantly improved the ability of 3D-GS to
model scenes with specular and anisotropic components without increasing the
number of 3D Gaussians. This improvement extends the applicability of 3D GS to
handle intricate scenarios with specular and anisotropic surfaces.