Semantic Text Compression for Classification
Emrecan Kutay Aylin Yener
We study semantic compression for text where meanings contained in the text
are conveyed to a source decoder, e.g., for classification. The main motivator
to move to such an approach of recovering the meaning without requiring exact
reconstruction is the potential resource savings, both in storage and in
conveying the information to another node. Towards this end, we propose
semantic quantization and compression approaches for text where we utilize
sentence embeddings and the semantic distortion metric to preserve the meaning.
Our results demonstrate that the proposed semantic approaches result in
substantial (orders of magnitude) savings in the required number of bits for
message representation at the expense of very modest accuracy loss compared to
the semantic agnostic baseline. We compare the results of proposed approaches
and observe that resource savings enabled by semantic quantization can be
further amplified by semantic clustering. Importantly, we observe the
generalizability of the proposed methodology which produces excellent results
on many benchmark text classification datasets with a diverse array of
contexts.