Tyche: Stochastic In-Context Learning for Medical Image Segmentation
Marianne Rakic Hallee E. Wong Jose Javier Gonzalez Ortiz Beth Cimini John Guttag Adrian V. Dalca
Existing learning-based solutions to medical image segmentation have two
important shortcomings. First, for most new segmentation task, a new model has
to be trained or fine-tuned. This requires extensive resources and machine
learning expertise, and is therefore often infeasible for medical researchers
and clinicians. Second, most existing segmentation methods produce a single
deterministic segmentation mask for a given image. In practice however, there
is often considerable uncertainty about what constitutes the correct
segmentation, and different expert annotators will often segment the same image
differently. We tackle both of these problems with Tyche, a model that uses a
context set to generate stochastic predictions for previously unseen tasks
without the need to retrain. Tyche differs from other in-context segmentation
methods in two important ways. (1) We introduce a novel convolution block
architecture that enables interactions among predictions. (2) We introduce
in-context test-time augmentation, a new mechanism to provide prediction
stochasticity. When combined with appropriate model design and loss functions,
Tyche can predict a set of plausible diverse segmentation candidates for new or
unseen medical images and segmentation tasks without the need to retrain.