my_awesome_qa_model

This model is a fine-tuned version of climatebert/distilroberta-base-climate-f on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.1299

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.002
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	1	5.5
No log	2.0	2	5.0
No log	3.0	3	4.3125
No log	4.0	4	3.625
No log	5.0	5	3.0625
No log	6.0	6	3.0
No log	7.0	7	2.6875
No log	8.0	8	2.3281
No log	9.0	9	2.0625
No log	10.0	10	1.8281
No log	11.0	11	1.6094
No log	12.0	12	1.3828
No log	13.0	13	1.2266
No log	14.0	14	1.0703
No log	15.0	15	0.9062
No log	16.0	16	0.7227
No log	17.0	17	0.5859
No log	18.0	18	0.4805
No log	19.0	19	0.4102
No log	20.0	20	0.3672
No log	21.0	21	0.3594
No log	22.0	22	0.3203
No log	23.0	23	0.2617
No log	24.0	24	0.2148
No log	25.0	25	0.1914
No log	26.0	26	0.1709
No log	27.0	27	0.1523
No log	28.0	28	0.1387
No log	29.0	29	0.1328
No log	30.0	30	0.1299

Framework versions

• PEFT 0.10.0
• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2