RoTaR: Efficient Row-Based Table Representation Learning via
  Teacher-Student Training
Zui Chen Lei Cao Sam Madden
We propose RoTaR, a row-based table representation learning method, to
address the efficiency and scalability issues faced by existing table
representation learning methods. The key idea of RoTaR is to generate
query-agnostic row representations that could be re-used via query-specific
aggregation. In addition to the row-based architecture, we introduce several
techniques: cell-aware position embedding, teacher-student training paradigm,
and selective backward to improve the performance of RoTaR model.