FusionSF: Fuse Heterogeneous Modalities in a Vector Quantized Framework
  for Robust Solar Power Forecasting
Ziqing Ma Wenwei Wang Tian Zhou Chao Chen Bingqing Peng Liang Sun Rong Jin
Accurate solar power forecasting is crucial to integrate photovoltaic plants
into the electric grid, schedule and secure the power grid safety. This problem
becomes more demanding for those newly installed solar plants which lack
sufficient data. Current research predominantly relies on historical solar
power data or numerical weather prediction in a single-modality format,
ignoring the complementary information provided in different modalities. In
this paper, we propose a multi-modality fusion framework to integrate
historical power data, numerical weather prediction, and satellite images,
significantly improving forecast performance. We introduce a vector quantized
framework that aligns modalities with varying information densities, striking a
balance between integrating sufficient information and averting model
overfitting. Our framework demonstrates strong zero-shot forecasting
capability, which is especially useful for those newly installed plants.
Moreover, we collect and release a multi-modal solar power (MMSP) dataset from
real-world plants to further promote the research of multi-modal solar
forecasting algorithms. Our extensive experiments show that our model not only
operates with robustness but also boosts accuracy in both zero-shot forecasting
and scenarios rich with training data, surpassing leading models. We have
incorporated it into our eForecaster platform and deployed it for more than 300
solar plants with a capacity of over 15GW.