A Primer on the Data Cleaning Pipeline
Rebecca C. Steorts
The availability of both structured and unstructured databases, such as
electronic health data, social media data, patent data, and surveys that are
often updated in real time, among others, has grown rapidly over the past
decade. With this expansion, the statistical and methodological questions
around data integration, or rather merging multiple data sources, has also
grown. Specifically, the science of the ``data cleaning pipeline'' contains
four stages that allow an analyst to perform downstream tasks, predictive
analyses, or statistical analyses on ``cleaned data.'' This article provides a
review of this emerging field, introducing technical terminology and commonly
used methods.