Complex-Valued Neural Networks for Data-Driven Signal Processing and
  Signal Understanding
Josiah W. Smith
Complex-valued neural networks have emerged boasting superior modeling
performance for many tasks across the signal processing, sensing, and
communications arenas. However, developing complex-valued models currently
demands development of basic deep learning operations, such as linear or
convolution layers, as modern deep learning frameworks like PyTorch and Tensor
flow do not adequately support complex-valued neural networks. This paper
overviews a package built on PyTorch with the intention of implementing
light-weight interfaces for common complex-valued neural network operations and
architectures. Similar to natural language understanding (NLU), which as
recently made tremendous leaps towards text-based intelligence, RF Signal
Understanding (RFSU) is a promising field extending conventional signal
processing algorithms using a hybrid approach of signal mechanics-based insight
with data-driven modeling power. Notably, we include efficient implementations
for linear, convolution, and attention modules in addition to activation
functions and normalization layers such as batchnorm and layernorm.
Additionally, we include efficient implementations of manifold-based
complex-valued neural network layers that have shown tremendous promise but
remain relatively unexplored in many research contexts. Although there is an
emphasis on 1-D data tensors, due to a focus on signal processing,
communications, and radar data, many of the routines are implemented for 2-D
and 3-D data as well. Specifically, the proposed approach offers a useful set
of tools and documentation for data-driven signal processing research and
practical implementation.