Hybrid deep learning and physics-based neural network for programmable
  illumination computational microscopy
Ruiqing Sun Delong Yang Shaohui Zhang Qun Hao
Relying on either deep models or physical models are two mainstream
approaches for solving inverse sample reconstruction problems in programmable
illumination computational microscopy. Solutions based on physical models
possess strong generalization capabilities while struggling with global
optimization of inverse problems due to a lack of insufficient physical
constraints. In contrast, deep learning methods have strong problem-solving
abilities, but their generalization ability is often questioned because of the
unclear physical principles. Besides, conventional deep models are difficult to
apply to some specific scenes because of the difficulty in acquiring
high-quality training data and their limited capacity to generalize across
different scenarios. In this paper, to combine the advantages of deep models
and physical models together, we propose a hybrid framework consisting of three
sub-neural networks (two deep learning networks and one physics-based network).
We first obtain a result with rich semantic information through a light deep
learning neural network and then use it as the initial value of the physical
network to make its output comply with physical process constraints. These two
results are then used as the input of a fusion deep learning neural work which
utilizes the paired features between the reconstruction results of two
different models to further enhance imaging quality. The final result
integrates the advantages of both deep models and physical models and can
quickly solve the computational reconstruction inverse problem in programmable
illumination computational microscopy and achieve better results. We verified
the feasibility and effectiveness of the proposed hybrid framework with
theoretical analysis and actual experiments on resolution targets and
biological samples.