Multitask Learning in Minimally Invasive Surgical Vision: A Review
Oluwatosin Alabi Tom Vercauteren Miaojing Shi
Minimally invasive surgery (MIS) has revolutionized many procedures and led
to reduced recovery time and risk of patient injury. However, MIS poses
additional complexity and burden on surgical teams. Data-driven surgical vision
algorithms are thought to be key building blocks in the development of future
MIS systems with improved autonomy. Recent advancements in machine learning and
computer vision have led to successful applications in analyzing videos
obtained from MIS with the promise of alleviating challenges in MIS videos.
Surgical scene and action understanding encompasses multiple related tasks
that, when solved individually, can be memory-intensive, inefficient, and fail
to capture task relationships. Multitask learning (MTL), a learning paradigm
that leverages information from multiple related tasks to improve performance
and aid generalization, is wellsuited for fine-grained and high-level
understanding of MIS data. This review provides an overview of the current
state-of-the-art MTL systems that leverage videos obtained from MIS. Beyond
listing published approaches, we discuss the benefits and limitations of these
MTL systems. Moreover, this manuscript presents an analysis of the literature
for various application fields of MTL in MIS, including those with large
models, highlighting notable trends, new directions of research, and
developments.