Animal Avatars: Reconstructing Animatable 3D Animals from Casual Videos
Remy Sabathier Niloy J. Mitra David Novotny
We present a method to build animatable dog avatars from monocular videos.
This is challenging as animals display a range of (unpredictable) non-rigid
movements and have a variety of appearance details (e.g., fur, spots, tails).
We develop an approach that links the video frames via a 4D solution that
jointly solves for animal's pose variation, and its appearance (in a canonical
pose). To this end, we significantly improve the quality of template-based
shape fitting by endowing the SMAL parametric model with Continuous Surface
Embeddings, which brings image-to-mesh reprojection constaints that are denser,
and thus stronger, than the previously used sparse semantic keypoint
correspondences. To model appearance, we propose an implicit duplex-mesh
texture that is defined in the canonical pose, but can be deformed using SMAL
pose coefficients and later rendered to enforce a photometric compatibility
with the input video frames. On the challenging CoP3D and APTv2 datasets, we
demonstrate superior results (both in terms of pose estimates and predicted
appearance) to existing template-free (RAC) and template-based approaches
(BARC, BITE).