Instant-NeRF: Instant On-Device Neural Radiance Field Training via
  Algorithm-Accelerator Co-Designed Near-Memory Processing
Yang Zhao Shang Wu Jingqun Zhang Sixu Li Chaojian Li Yingyan Lin
Instant on-device Neural Radiance Fields (NeRFs) are in growing demand for
unleashing the promise of immersive AR/VR experiences, but are still limited by
their prohibitive training time. Our profiling analysis reveals a memory-bound
inefficiency in NeRF training. To tackle this inefficiency, near-memory
processing (NMP) promises to be an effective solution, but also faces
challenges due to the unique workloads of NeRFs, including the random hash
table lookup, random point processing sequence, and heterogeneous bottleneck
steps. Therefore, we propose the first NMP framework, Instant-NeRF, dedicated
to enabling instant on-device NeRF training. Experiments on eight datasets
consistently validate the effectiveness of Instant-NeRF.