mt5-small-finetuned-xlsum-zh-en

This model is a fine-tuned version of google/mt5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.4765
• Rouge1: 13.6815
• Rouge2: 1.9963
• Rougel: 11.1618
• Rougelsum: 11.196

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
4.179	1.0	175	3.4764	12.6283	1.9745	10.3319	10.3779
3.9528	2.0	350	3.4743	13.3663	1.992	11.0757	11.0275
3.8472	3.0	525	3.4887	12.8037	1.8678	10.3381	10.3357
3.7711	4.0	700	3.4765	13.6815	1.9963	11.1618	11.196
3.7389	5.0	875	3.4853	13.1565	1.9543	10.6958	10.7191
3.7368	6.0	1050	3.4717	13.025	1.9673	10.5016	10.5047
3.7475	7.0	1225	3.4678	12.7763	1.8506	10.3091	10.3242
3.783	8.0	1400	3.4659	12.9145	1.9185	10.3757	10.4012

Framework versions

• Transformers 4.47.1
• Pytorch 2.2.2+cu121
• Datasets 3.2.0
• Tokenizers 0.21.0