Implicit Neural Representations and the Algebra of Complex Wavelets
T. Mitchell Roddenberry Vishwanath Saragadam Maarten V. de Hoop Richard G. Baraniuk
Implicit neural representations (INRs) have arisen as useful methods for
representing signals on Euclidean domains. By parameterizing an image as a
multilayer perceptron (MLP) on Euclidean space, INRs effectively represent
signals in a way that couples spatial and spectral features of the signal that
is not obvious in the usual discrete representation, paving the way for
continuous signal processing and machine learning approaches that were not
previously possible. Although INRs using sinusoidal activation functions have
been studied in terms of Fourier theory, recent works have shown the advantage
of using wavelets instead of sinusoids as activation functions, due to their
ability to simultaneously localize in both frequency and space. In this work,
we approach such INRs and demonstrate how they resolve high-frequency features
of signals from coarse approximations done in the first layer of the MLP. This
leads to multiple prescriptions for the design of INR architectures, including
the use of complex wavelets, decoupling of low and band-pass approximations,
and initialization schemes based on the singularities of the desired signal.