Data-driven Crop Growth Simulation on Time-varying Generated Images
  using Multi-conditional Generative Adversarial Networks
Lukas Drees Dereje T. Demie Madhuri R. Paul Johannes Leonhardt Sabine J. Seidel Thomas F. Döring Ribana Roscher
Image-based crop growth modeling can substantially contribute to precision
agriculture by revealing spatial crop development over time, which allows an
early and location-specific estimation of relevant future plant traits, such as
leaf area or biomass. A prerequisite for realistic and sharp crop image
generation is the integration of multiple growth-influencing conditions in a
model, such as an image of an initial growth stage, the associated growth time,
and further information about the field treatment. We present a two-stage
framework consisting first of an image prediction model and second of a growth
estimation model, which both are independently trained. The image prediction
model is a conditional Wasserstein generative adversarial network (CWGAN). In
the generator of this model, conditional batch normalization (CBN) is used to
integrate different conditions along with the input image. This allows the
model to generate time-varying artificial images dependent on multiple
influencing factors of different kinds. These images are used by the second
part of the framework for plant phenotyping by deriving plant-specific traits
and comparing them with those of non-artificial (real) reference images. For
various crop datasets, the framework allows realistic, sharp image predictions
with a slight loss of quality from short-term to long-term predictions.
Simulations of varying growth-influencing conditions performed with the trained
framework provide valuable insights into how such factors relate to crop
appearances, which is particularly useful in complex, less explored crop
mixture systems. Further results show that adding process-based simulated
biomass as a condition increases the accuracy of the derived phenotypic traits
from the predicted images. This demonstrates the potential of our framework to
serve as an interface between an image- and process-based crop growth model.