Differentiable Biomechanics Unlocks Opportunities for Markerless Motion
  Capture
R. James Cotton
Recent developments have created differentiable physics simulators designed
for machine learning pipelines that can be accelerated on a GPU. While these
can simulate biomechanical models, these opportunities have not been exploited
for biomechanics research or markerless motion capture. We show that these
simulators can be used to fit inverse kinematics to markerless motion capture
data, including scaling the model to fit the anthropomorphic measurements of an
individual. This is performed end-to-end with an implicit representation of the
movement trajectory, which is propagated through the forward kinematic model to
minimize the error from the 3D markers reprojected into the images. The
differential optimizer yields other opportunities, such as adding bundle
adjustment during trajectory optimization to refine the extrinsic camera
parameters or meta-optimization to improve the base model jointly over
trajectories from multiple participants. This approach improves the
reprojection error from markerless motion capture over prior methods and
produces accurate spatial step parameters compared to an instrumented walkway
for control and clinical populations.