SIDAR: Synthetic Image Dataset for Alignment & Restoration
Monika Kwiatkowski Simon Matern Olaf Hellwich
Image alignment and image restoration are classical computer vision tasks.
However, there is still a lack of datasets that provide enough data to train
and evaluate end-to-end deep learning models. Obtaining ground-truth data for
image alignment requires sophisticated structure-from-motion methods or optical
flow systems that often do not provide enough data variance, i.e., typically
providing a high number of image correspondences, while only introducing few
changes of scenery within the underlying image sequences. Alternative
approaches utilize random perspective distortions on existing image data.
However, this only provides trivial distortions, lacking the complexity and
variance of real-world scenarios. Instead, our proposed data augmentation helps
to overcome the issue of data scarcity by using 3D rendering: images are added
as textures onto a plane, then varying lighting conditions, shadows, and
occlusions are added to the scene. The scene is rendered from multiple
viewpoints, generating perspective distortions more consistent with real-world
scenarios, with homographies closely resembling those of camera projections
rather than randomized homographies. For each scene, we provide a sequence of
distorted images with corresponding occlusion masks, homographies, and
ground-truth labels. The resulting dataset can serve as a training and
evaluation set for a multitude of tasks involving image alignment and artifact
removal, such as deep homography estimation, dense image matching, 2D bundle
adjustment, inpainting, shadow removal, denoising, content retrieval, and
background subtraction. Our data generation pipeline is customizable and can be
applied to any existing dataset, serving as a data augmentation to further
improve the feature learning of any existing method.