What Happens to a Dataset Transformed by a Projection-based Concept
  Removal Method?
Richard Johansson
We investigate the behavior of methods that use linear projections to remove
information about a concept from a language representation, and we consider the
question of what happens to a dataset transformed by such a method. A
theoretical analysis and experiments on real-world and synthetic data show that
these methods inject strong statistical dependencies into the transformed
datasets. After applying such a method, the representation space is highly
structured: in the transformed space, an instance tends to be located near
instances of the opposite label. As a consequence, the original labeling can in
some cases be reconstructed by applying an anti-clustering method.