Flan-T5-finetuned-cti2_NER-RE_v1_0_0_full-bits-1121

This model is a fine-tuned version of HoangCuongNguyen/Flan-T5-finetuned-cti2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1824
• Rouge1: 65.2986
• Rouge2: 58.7857
• Rougel: 64.8007
• Rougelsum: 64.8791
• Gen Len: 18.9271

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.03
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.5677	1.0	430	0.4877	55.0379	39.7155	52.8487	52.9528	18.9062
0.4359	2.0	860	0.2844	59.6863	50.2141	58.7456	59.0242	18.9688
0.1506	3.0	1290	0.1983	63.4922	55.1698	63.0887	63.1894	18.9271
0.2049	4.0	1720	0.1784	64.731	58.0105	64.2883	64.3686	18.9479
0.126	5.0	2150	0.1824	65.2986	58.7857	64.8007	64.8791	18.9271

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu117
• Datasets 2.15.0
• Tokenizers 0.13.3