APTv2: Benchmarking Animal Pose Estimation and Tracking with a
  Large-scale Dataset and Beyond
Yuxiang Yang Yingqi Deng Yufei Xu Jing Zhang
Animal Pose Estimation and Tracking (APT) is a critical task in detecting and
monitoring the keypoints of animals across a series of video frames, which is
essential for understanding animal behavior. Past works relating to animals
have primarily focused on either animal tracking or single-frame animal pose
estimation only, neglecting the integration of both aspects. The absence of
comprehensive APT datasets inhibits the progression and evaluation of animal
pose estimation and tracking methods based on videos, thereby constraining
their real-world applications. To fill this gap, we introduce APTv2, the
pioneering large-scale benchmark for animal pose estimation and tracking. APTv2
comprises 2,749 video clips filtered and collected from 30 distinct animal
species. Each video clip includes 15 frames, culminating in a total of 41,235
frames. Following meticulous manual annotation and stringent verification, we
provide high-quality keypoint and tracking annotations for a total of 84,611
animal instances, split into easy and hard subsets based on the number of
instances that exists in the frame. With APTv2 as the foundation, we establish
a simple baseline method named \posetrackmethodname and provide benchmarks for
representative models across three tracks: (1) single-frame animal pose
estimation track to evaluate both intra- and inter-domain transfer learning
performance, (2) low-data transfer and generalization track to evaluate the
inter-species domain generalization performance, and (3) animal pose tracking
track. Our experimental results deliver key empirical insights, demonstrating
that APTv2 serves as a valuable benchmark for animal pose estimation and
tracking. It also presents new challenges and opportunities for future
research. The code and dataset are released at
\href{https://github.com/ViTAE-Transformer/APTv2}{https://github.com/ViTAE-Transformer/APTv2}.