VR-GS: A Physical Dynamics-Aware Interactive Gaussian Splatting System
  in Virtual Reality
Ying Jiang Chang Yu Tianyi Xie Xuan Li Yutao Feng Huamin Wang Minchen Li Henry Lau Feng Gao Yin Yang Chenfanfu Jiang
As consumer Virtual Reality (VR) and Mixed Reality (MR) technologies gain
momentum, there's a growing focus on the development of engagements with 3D
virtual content. Unfortunately, traditional techniques for content creation,
editing, and interaction within these virtual spaces are fraught with
difficulties. They tend to be not only engineering-intensive but also require
extensive expertise, which adds to the frustration and inefficiency in virtual
object manipulation. Our proposed VR-GS system represents a leap forward in
human-centered 3D content interaction, offering a seamless and intuitive user
experience. By developing a physical dynamics-aware interactive Gaussian
Splatting in a Virtual Reality setting, and constructing a highly efficient
two-level embedding strategy alongside deformable body simulations, VR-GS
ensures real-time execution with highly realistic dynamic responses. The
components of our Virtual Reality system are designed for high efficiency and
effectiveness, starting from detailed scene reconstruction and object
segmentation, advancing through multi-view image in-painting, and extending to
interactive physics-based editing. The system also incorporates real-time
deformation embedding and dynamic shadow casting, ensuring a comprehensive and
engaging virtual experience.Our project page is available at:
https://yingjiang96.github.io/VR-GS/.