Studying the Impact of Stochasticity on the Evaluation of Deep Neural
  Networks for Forest-Fire Prediction
Harshit Kumar Biswadeep Chakraborty Beomseok Kang Saibal Mukhopadhyay
This paper presents the first systematic study of the evaluation of Deep
Neural Networks (DNNs) for discrete dynamical systems under stochastic
assumptions, with a focus on wildfire prediction. We develop a framework to
study the impact of stochasticity on two classes of evaluation metrics:
classification-based metrics, which assess fidelity to observed ground truth
(GT), and proper scoring rules, which test fidelity-to-statistic. Our findings
reveal that evaluating for fidelity-to-statistic is a reliable alternative in
highly stochastic scenarios. We extend our analysis to real-world wildfire
data, highlighting limitations in traditional wildfire prediction evaluation
methods, and suggest interpretable stochasticity-compatible alternatives.