ViFiT: Reconstructing Vision Trajectories from IMU and Wi-Fi Fine Time
  Measurements
Bryan Bo Cao Abrar Alali Hansi Liu Nicholas Meegan Marco Gruteser Kristin Dana Ashwin Ashok Shubham Jain
Tracking subjects in videos is one of the most widely used functions in
camera-based IoT applications such as security surveillance, smart city traffic
safety enhancement, vehicle to pedestrian communication and so on. In the
computer vision domain, tracking is usually achieved by first detecting
subjects with bounding boxes, then associating detected bounding boxes across
video frames. For many IoT systems, images captured by cameras are usually sent
over the network to be processed at a different site that has more powerful
computing resources than edge devices. However, sending entire frames through
the network causes significant bandwidth consumption that may exceed the system
bandwidth constraints. To tackle this problem, we propose ViFiT, a
transformer-based model that reconstructs vision bounding box trajectories from
phone data (IMU and Fine Time Measurements). It leverages a transformer ability
of better modeling long-term time series data. ViFiT is evaluated on Vi-Fi
Dataset, a large-scale multimodal dataset in 5 diverse real world scenes,
including indoor and outdoor environments. To fill the gap of proper metrics of
jointly capturing the system characteristics of both tracking quality and video
bandwidth reduction, we propose a novel evaluation framework dubbed Minimum
Required Frames (MRF) and Minimum Required Frames Ratio (MRFR). ViFiT achieves
an MRFR of 0.65 that outperforms the state-of-the-art approach for cross-modal
reconstruction in LSTM Encoder-Decoder architecture X-Translator of 0.98,
resulting in a high frame reduction rate as 97.76%.