Gaussian Splatting on the Move: Blur and Rolling Shutter Compensation
  for Natural Camera Motion
Otto Seiskari Jerry Ylilammi Valtteri Kaatrasalo Pekka Rantalankila Matias Turkulainen Juho Kannala Esa Rahtu Arno Solin
High-quality scene reconstruction and novel view synthesis based on Gaussian
Splatting (3DGS) typically require steady, high-quality photographs, often
impractical to capture with handheld cameras. We present a method that adapts
to camera motion and allows high-quality scene reconstruction with handheld
video data suffering from motion blur and rolling shutter distortion. Our
approach is based on detailed modelling of the physical image formation process
and utilizes velocities estimated using visual-inertial odometry (VIO). Camera
poses are considered non-static during the exposure time of a single image
frame and camera poses are further optimized in the reconstruction process. We
formulate a differentiable rendering pipeline that leverages screen space
approximation to efficiently incorporate rolling-shutter and motion blur
effects into the 3DGS framework. Our results with both synthetic and real data
demonstrate superior performance in mitigating camera motion over existing
methods, thereby advancing 3DGS in naturalistic settings.