L3Cube-IndicNews: News-based Short Text and Long Document Classification
  Datasets in Indic Languages
Aishwarya Mirashi Srushti Sonavane Purva Lingayat Tejas Padhiyar Raviraj Joshi
In this work, we introduce L3Cube-IndicNews, a multilingual text
classification corpus aimed at curating a high-quality dataset for Indian
regional languages, with a specific focus on news headlines and articles. We
have centered our work on 10 prominent Indic languages, including Hindi,
Bengali, Marathi, Telugu, Tamil, Gujarati, Kannada, Odia, Malayalam, and
Punjabi. Each of these news datasets comprises 10 or more classes of news
articles. L3Cube-IndicNews offers 3 distinct datasets tailored to handle
different document lengths that are classified as: Short Headlines
Classification (SHC) dataset containing the news headline and news category,
Long Document Classification (LDC) dataset containing the whole news article
and the news category, and Long Paragraph Classification (LPC) containing
sub-articles of the news and the news category. We maintain consistent labeling
across all 3 datasets for in-depth length-based analysis. We evaluate each of
these Indic language datasets using 4 different models including monolingual
BERT, multilingual Indic Sentence BERT (IndicSBERT), and IndicBERT. This
research contributes significantly to expanding the pool of available text
classification datasets and also makes it possible to develop topic
classification models for Indian regional languages. This also serves as an
excellent resource for cross-lingual analysis owing to the high overlap of
labels among languages. The datasets and models are shared publicly at
https://github.com/l3cube-pune/indic-nlp