SSL Framework for Causal Inconsistency between Structures and
  Representations
Hang Chen Xinyu Yang Keqing Du
The cross-pollination of deep learning and causal discovery has catalyzed a
burgeoning field of research seeking to elucidate causal relationships within
non-statistical data forms like images, videos, and text. Such data, often
being named `indefinite data', exhibit unique challenges-inconsistency between
causal structure and representation, which are not common in conventional data
forms. To tackle this issue, we theoretically develop intervention strategies
suitable for indefinite data and derive causal consistency condition (CCC).
Moreover, we design a self-supervised learning (SSL) framework that considers
interventions as `views' and CCC as a `philosophy' with two implement examples
on Supervised Specialized Models (SSMs) and Large Language Models (LLMs),
respectively. To evaluate pure inconsistency manifestations, we have prepared
the first high-quality causal dialogue dataset-Causalogue. Evaluations are also
performed on three other downstream tasks. Extensive experimentation has
substantiated the efficacy of our methodology, illuminating how CCC could
potentially play an influential role in various fields.