htlou/backup_0202_llamafactory_AA_preference_Cherry_0_60-llava-mistral

AA_preference_Cherry_0_60

This model is a fine-tuned version of llava-hf/llava-v1.6-mistral-7b-hf on the AA_preference_Cherry_0_60 dataset. It achieves the following results on the evaluation set:

• Loss: 0.4983
• Rewards/chosen: 2.3376
• Rewards/rejected: -0.8230
• Rewards/accuracies: 0.8074
• Rewards/margins: 3.1606
• Logps/rejected: -240.8466
• Logps/chosen: -306.5101
• Logits/rejected: -2.0946
• Logits/chosen: -2.1213

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 8
• gradient_accumulation_steps: 4
• total_train_batch_size: 256
• total_eval_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 10
• num_epochs: 3.0

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.5436	0.4923	40	0.5011	2.5953	0.9447	0.7736	1.6506	-223.1691	-303.9334	-2.0088	-2.0351
0.4689	0.9846	80	0.4769	1.8328	-0.2925	0.8074	2.1253	-235.5412	-311.5584	-2.2292	-2.2518
0.1825	1.4769	120	0.5121	1.9848	-0.7992	0.8142	2.7840	-240.6085	-310.0383	-2.1958	-2.2158
0.2112	1.9692	160	0.4885	2.4604	-0.3263	0.8176	2.7867	-235.8799	-305.2829	-2.1603	-2.1813
0.1059	2.4615	200	0.4947	2.3506	-0.7274	0.8108	3.0780	-239.8905	-306.3801	-2.1061	-2.1317
0.1134	2.9538	240	0.4985	2.3366	-0.8241	0.8108	3.1607	-240.8571	-306.5204	-2.0945	-2.1211

Framework versions

• Transformers 4.45.2
• Pytorch 2.4.0+cu121
• Datasets 2.21.0
• Tokenizers 0.20.3