Neural Radiance Fields for Transparent Object Using Visual Hull
Heechan Yoon Seungkyu Lee
Unlike opaque object, novel view synthesis of transparent object is a
challenging task, because transparent object refracts light of background
causing visual distortions on the transparent object surface along the
viewpoint change. Recently introduced Neural Radiance Fields (NeRF) is a view
synthesis method. Thanks to its remarkable performance improvement, lots of
following applications based on NeRF in various topics have been developed.
However, if an object with a different refractive index is included in a scene
such as transparent object, NeRF shows limited performance because refracted
light ray at the surface of the transparent object is not appropriately
considered. To resolve the problem, we propose a NeRF-based method consisting
of the following three steps: First, we reconstruct a three-dimensional shape
of a transparent object using visual hull. Second, we simulate the refraction
of the rays inside of the transparent object according to Snell's law. Last, we
sample points through refracted rays and put them into NeRF. Experimental
evaluation results demonstrate that our method addresses the limitation of
conventional NeRF with transparent objects.