Blendshapes GHUM: Real-time Monocular Facial Blendshape Prediction
Ivan Grishchenko Geng Yan Eduard Gabriel Bazavan Andrei Zanfir Nikolai Chinaev Karthik Raveendran Matthias Grundmann Cristian Sminchisescu
We present Blendshapes GHUM, an on-device ML pipeline that predicts 52 facial
blendshape coefficients at 30+ FPS on modern mobile phones, from a single
monocular RGB image and enables facial motion capture applications like virtual
avatars. Our main contributions are: i) an annotation-free offline method for
obtaining blendshape coefficients from real-world human scans, ii) a
lightweight real-time model that predicts blendshape coefficients based on
facial landmarks.