Automated Conversion of Music Videos into Lyric Videos
Jiaju Ma Anyi Rao Li-Yi Wei Rubaiat Habib Kazi Hijung Valentina Shin Maneesh Agrawala
Musicians and fans often produce lyric videos, a form of music videos that
showcase the song's lyrics, for their favorite songs. However, making such
videos can be challenging and time-consuming as the lyrics need to be added in
synchrony and visual harmony with the video. Informed by prior work and close
examination of existing lyric videos, we propose a set of design guidelines to
help creators make such videos. Our guidelines ensure the readability of the
lyric text while maintaining a unified focus of attention. We instantiate these
guidelines in a fully automated pipeline that converts an input music video
into a lyric video. We demonstrate the robustness of our pipeline by generating
lyric videos from a diverse range of input sources. A user study shows that
lyric videos generated by our pipeline are effective in maintaining text
readability and unifying the focus of attention.