Semi-Supervised Coupled Thin-Plate Spline Model for Rotation Correction
  and Beyond
Lang Nie Chunyu Lin Kang Liao Shuaicheng Liu Yao Zhao
Thin-plate spline (TPS) is a principal warp that allows for representing
elastic, nonlinear transformation with control point motions. With the increase
of control points, the warp becomes increasingly flexible but usually
encounters a bottleneck caused by undesired issues, e.g., content distortion.
In this paper, we explore generic applications of TPS in single-image-based
warping tasks, such as rotation correction, rectangling, and portrait
correction. To break this bottleneck, we propose the coupled thin-plate spline
model (CoupledTPS), which iteratively couples multiple TPS with limited control
points into a more flexible and powerful transformation. Concretely, we first
design an iterative search to predict new control points according to the
current latent condition. Then, we present the warping flow as a bridge for the
coupling of different TPS transformations, effectively eliminating
interpolation errors caused by multiple warps. Besides, in light of the
laborious annotation cost, we develop a semi-supervised learning scheme to
improve warping quality by exploiting unlabeled data. It is formulated through
dual transformation between the searched control points of unlabeled data and
its graphic augmentation, yielding an implicit correction consistency
constraint. Finally, we collect massive unlabeled data to exhibit the benefit
of our semi-supervised scheme in rotation correction. Extensive experiments
demonstrate the superiority and universality of CoupledTPS over the existing
state-of-the-art (SoTA) solutions for rotation correction and beyond. The code
and data will be available at https://github.com/nie-lang/CoupledTPS.