A Sampling Theory Perspective on Activations for Implicit Neural
  Representations
Hemanth Saratchandran Sameera Ramasinghe Violetta Shevchenko Alexander Long Simon Lucey
Implicit Neural Representations (INRs) have gained popularity for encoding
signals as compact, differentiable entities. While commonly using techniques
like Fourier positional encodings or non-traditional activation functions
(e.g., Gaussian, sinusoid, or wavelets) to capture high-frequency content,
their properties lack exploration within a unified theoretical framework.
Addressing this gap, we conduct a comprehensive analysis of these activations
from a sampling theory perspective. Our investigation reveals that sinc
activations, previously unused in conjunction with INRs, are theoretically
optimal for signal encoding. Additionally, we establish a connection between
dynamical systems and INRs, leveraging sampling theory to bridge these two
paradigms.