Transfer: Cross Modality Knowledge Transfer using Adversarial Networks
  -- A Study on Gesture Recognition
Payal Kamboj Ayan Banerjee Sandeep K. S. Gupta
Knowledge transfer across sensing technology is a novel concept that has been
recently explored in many application domains, including gesture-based human
computer interaction. The main aim is to gather semantic or data driven
information from a source technology to classify / recognize instances of
unseen classes in the target technology. The primary challenge is the
significant difference in dimensionality and distribution of feature sets
between the source and the target technologies. In this paper, we propose
TRANSFER, a generic framework for knowledge transfer between a source and a
target technology. TRANSFER uses a language-based representation of a hand
gesture, which captures a temporal combination of concepts such as handshape,
location, and movement that are semantically related to the meaning of a word.
By utilizing a pre-specified syntactic structure and tokenizer, TRANSFER
segments a hand gesture into tokens and identifies individual components using
a token recognizer. The tokenizer in this language-based recognition system
abstracts the low-level technology-specific characteristics to the machine
interface, enabling the design of a discriminator that learns
technology-invariant features essential for recognition of gestures in both
source and target technologies. We demonstrate the usage of TRANSFER for three
different scenarios: a) transferring knowledge across technology by learning
gesture models from video and recognizing gestures using WiFi, b) transferring
knowledge from video to accelerometer, and d) transferring knowledge from
accelerometer to WiFi signals.