MEMTRACK: A Deep Learning-Based Approach to Microrobot Tracking in Dense
  and Low-Contrast Environments
Medha Sawhney Bhas Karmarkar Eric J. Leaman Arka Daw Anuj Karpatne Bahareh Behkam
Tracking microrobots is challenging, considering their minute size and high
speed. As the field progresses towards developing microrobots for biomedical
applications and conducting mechanistic studies in physiologically relevant
media (e.g., collagen), this challenge is exacerbated by the dense surrounding
environments with feature size and shape comparable to microrobots. Herein, we
report Motion Enhanced Multi-level Tracker (MEMTrack), a robust pipeline for
detecting and tracking microrobots using synthetic motion features, deep
learning-based object detection, and a modified Simple Online and Real-time
Tracking (SORT) algorithm with interpolation for tracking. Our object detection
approach combines different models based on the object's motion pattern. We
trained and validated our model using bacterial micro-motors in collagen
(tissue phantom) and tested it in collagen and aqueous media. We demonstrate
that MEMTrack accurately tracks even the most challenging bacteria missed by
skilled human annotators, achieving precision and recall of 77% and 48% in
collagen and 94% and 35% in liquid media, respectively. Moreover, we show that
MEMTrack can quantify average bacteria speed with no statistically significant
difference from the laboriously-produced manual tracking data. MEMTrack
represents a significant contribution to microrobot localization and tracking,
and opens the potential for vision-based deep learning approaches to microrobot
control in dense and low-contrast settings. All source code for training and
testing MEMTrack and reproducing the results of the paper have been made
publicly available https://github.com/sawhney-medha/MEMTrack.