Streaming Sequence Transduction through Dynamic Compression
Weiting Tan Yunmo Chen Tongfei Chen Guanghui Qin Haoran Xu Heidi C. Zhang Benjamin Van Durme Philipp Koehn
We introduce STAR (Stream Transduction with Anchor Representations), a novel
Transformer-based model designed for efficient sequence-to-sequence
transduction over streams. STAR dynamically segments input streams to create
compressed anchor representations, achieving nearly lossless compression (12x)
in Automatic Speech Recognition (ASR) and outperforming existing methods.
Moreover, STAR demonstrates superior segmentation and latency-quality
trade-offs in simultaneous speech-to-text tasks, optimizing latency, memory
footprint, and quality.