Learning Multimodal Volumetric Features for Large-Scale Neuron Tracing
Qihua Chen Xuejin Chen Chenxuan Wang Yixiong Liu Zhiwei Xiong Feng Wu
The current neuron reconstruction pipeline for electron microscopy (EM) data
usually includes automatic image segmentation followed by extensive human
expert proofreading. In this work, we aim to reduce human workload by
predicting connectivity between over-segmented neuron pieces, taking both
microscopy image and 3D morphology features into account, similar to human
proofreading workflow. To this end, we first construct a dataset, named
FlyTracing, that contains millions of pairwise connections of segments
expanding the whole fly brain, which is three orders of magnitude larger than
existing datasets for neuron segment connection. To learn sophisticated
biological imaging features from the connectivity annotations, we propose a
novel connectivity-aware contrastive learning method to generate dense
volumetric EM image embedding. The learned embeddings can be easily
incorporated with any point or voxel-based morphological representations for
automatic neuron tracing. Extensive comparisons of different combination
schemes of image and morphological representation in identifying split errors
across the whole fly brain demonstrate the superiority of the proposed
approach, especially for the locations that contain severe imaging artifacts,
such as section missing and misalignment. The dataset and code are available at
https://github.com/Levishery/Flywire-Neuron-Tracing.