FinTree: Financial Dataset Pretrain Transformer Encoder for Relation
  Extraction
Hyunjong Ok
We present FinTree, Financial Dataset Pretrain Transformer Encoder for
Relation Extraction. Utilizing an encoder language model, we further pretrain
FinTree on the financial dataset, adapting the model in financial domain tasks.
FinTree stands out with its novel structure that predicts a masked token
instead of the conventional [CLS] token, inspired by the Pattern Exploiting
Training methodology. This structure allows for more accurate relation
predictions between two given entities. The model is trained with a unique
input pattern to provide contextual and positional information about the
entities of interest, and a post-processing step ensures accurate predictions
in line with the entity types. Our experiments demonstrate that FinTree
outperforms on the REFinD, a large-scale financial relation extraction dataset.
The code and pretrained models are available at
https://github.com/HJ-Ok/FinTree.