Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey
  with Causality Perspectives
Haoyang Liu Maheep Chaudhary Haohan Wang
The trustworthiness of machine learning has emerged as a critical topic in
the field, encompassing various applications and research areas such as
robustness, security, interpretability, and fairness. The last decade saw the
development of numerous methods addressing these challenges. In this survey, we
systematically review these advancements from a data-centric perspective,
highlighting the shortcomings of traditional empirical risk minimization (ERM)
training in handling challenges posed by the data.
  Interestingly, we observe a convergence of these methods, despite being
developed independently across trustworthy machine learning subfields. Pearl's
hierarchy of causality offers a unifying framework for these techniques.
Accordingly, this survey presents the background of trustworthy machine
learning development using a unified set of concepts, connects this language to
Pearl's causal hierarchy, and finally discusses methods explicitly inspired by
causality literature. We provide a unified language with mathematical
vocabulary to link these methods across robustness, adversarial robustness,
interpretability, and fairness, fostering a more cohesive understanding of the
field.
  Further, we explore the trustworthiness of large pretrained models. After
summarizing dominant techniques like fine-tuning, parameter-efficient
fine-tuning, prompting, and reinforcement learning with human feedback, we draw
connections between them and the standard ERM. This connection allows us to
build upon the principled understanding of trustworthy methods, extending it to
these new techniques in large pretrained models, paving the way for future
methods. Existing methods under this perspective are also reviewed.
  Lastly, we offer a brief summary of the applications of these methods and
discuss potential future aspects related to our survey. For more information,
please visit http://trustai.one.