Improved Training for End-to-End Streaming Automatic Speech Recognition
  Model with Punctuation
Hanbyul Kim Seunghyun Seo Lukas Lee Seolki Baek
Punctuated text prediction is crucial for automatic speech recognition as it
enhances readability and impacts downstream natural language processing tasks.
In streaming scenarios, the ability to predict punctuation in real-time is
particularly desirable but presents a difficult technical challenge. In this
work, we propose a method for predicting punctuated text from input speech
using a chunk-based Transformer encoder trained with Connectionist Temporal
Classification (CTC) loss. The acoustic model trained with long sequences by
concatenating the input and target sequences can learn punctuation marks
attached to the end of sentences more effectively. Additionally, by combining
CTC losses on the chunks and utterances, we achieved both the improved F1 score
of punctuation prediction and Word Error Rate (WER).