Domain-Specific Language Model Post-Training for Indonesian Financial
  NLP
Ni Putu Intan Maharani Yoga Yustiawan Fauzy Caesar Rochim Ayu Purwarianti
BERT and IndoBERT have achieved impressive performance in several NLP tasks.
There has been several investigation on its adaption in specialized domains
especially for English language. We focus on financial domain and Indonesian
language, where we perform post-training on pre-trained IndoBERT for financial
domain using a small scale of Indonesian financial corpus. In this paper, we
construct an Indonesian self-supervised financial corpus, Indonesian financial
sentiment analysis dataset, Indonesian financial topic classification dataset,
and release a family of BERT models for financial NLP. We also evaluate the
effectiveness of domain-specific post-training on sentiment analysis and topic
classification tasks. Our findings indicate that the post-training increases
the effectiveness of a language model when it is fine-tuned to domain-specific
downstream tasks.