Data Pipeline Training: Integrating AutoML to Optimize the Data Flow of
  Machine Learning Models
Jiang Wu Hongbo Wang Chunhe Ni Chenwei Zhang Wenran Lu
Data Pipeline plays an indispensable role in tasks such as modeling machine
learning and developing data products. With the increasing diversification and
complexity of Data sources, as well as the rapid growth of data volumes,
building an efficient Data Pipeline has become crucial for improving work
efficiency and solving complex problems. This paper focuses on exploring how to
optimize data flow through automated machine learning methods by integrating
AutoML with Data Pipeline. We will discuss how to leverage AutoML technology to
enhance the intelligence of Data Pipeline, thereby achieving better results in
machine learning tasks. By delving into the automation and optimization of Data
flows, we uncover key strategies for constructing efficient data pipelines that
can adapt to the ever-changing data landscape. This not only accelerates the
modeling process but also provides innovative solutions to complex problems,
enabling more significant outcomes in increasingly intricate data domains.
Keywords- Data Pipeline Training;AutoML; Data environment; Machine learning