Toward a Better Understanding of Fourier Neural Operators: Analysis and
  Improvement from a Spectral Perspective
Shaoxiang Qin Fuyuan Lyu Wenhui Peng Dingyang Geng Ju Wang Naiping Gao Xue Liu Liangzhu Leon Wang
In solving partial differential equations (PDEs), Fourier Neural Operators
(FNOs) have exhibited notable effectiveness compared to Convolutional Neural
Networks (CNNs). This paper presents clear empirical evidence through spectral
analysis to elucidate the superiority of FNO over CNNs: FNO is significantly
more capable of learning low-frequencies. This empirical evidence also unveils
FNO's distinct low-frequency bias, which limits FNO's effectiveness in learning
high-frequency information from PDE data. To tackle this challenge, we
introduce SpecBoost, an ensemble learning framework that employs multiple FNOs
to better capture high-frequency information. Specifically, a secondary FNO is
utilized to learn the overlooked high-frequency information from the prediction
residual of the initial FNO. Experiments demonstrate that SpecBoost noticeably
enhances FNO's prediction accuracy on diverse PDE applications, achieving an up
to 71% improvement.