Canonical Factors for Hybrid Neural Fields
Brent Yi Weijia Zeng Sam Buchanan Yi Ma
Factored feature volumes offer a simple way to build more compact, efficient,
and intepretable neural fields, but also introduce biases that are not
necessarily beneficial for real-world data. In this work, we (1) characterize
the undesirable biases that these architectures have for axis-aligned signals
-- they can lead to radiance field reconstruction differences of as high as 2
PSNR -- and (2) explore how learning a set of canonicalizing transformations
can improve representations by removing these biases. We prove in a
two-dimensional model problem that simultaneously learning these
transformations together with scene appearance succeeds with drastically
improved efficiency. We validate the resulting architectures, which we call
TILTED, using image, signed distance, and radiance field reconstruction tasks,
where we observe improvements across quality, robustness, compactness, and
runtime. Results demonstrate that TILTED can enable capabilities comparable to
baselines that are 2x larger, while highlighting weaknesses of neural field
evaluation procedures.