em

This model is a fine-tuned version of microsoft/deberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0866
• Exact Match Accuracy: 0.4247
• Precision Micro: 0.7071
• Recall Micro: 0.4573
• F1 Micro: 0.5554
• Precision Macro: 0.5018
• Recall Macro: 0.3205
• F1 Macro: 0.3658
• Classification Report: {'admiration': {'precision': 0.6727941176470589, 'recall': 0.7261904761904762, 'f1-score': 0.6984732824427481, 'support': 504.0}, 'amusement': {'precision': 0.7751677852348994, 'recall': 0.875, 'f1-score': 0.8220640569395018, 'support': 264.0}, 'anger': {'precision': 0.5193798449612403, 'recall': 0.3383838383838384, 'f1-score': 0.40978593272171254, 'support': 198.0}, 'annoyance': {'precision': 0.5454545454545454, 'recall': 0.05625, 'f1-score': 0.10198300283286119, 'support': 320.0}, 'approval': {'precision': 0.6413793103448275, 'recall': 0.26495726495726496, 'f1-score': 0.375, 'support': 351.0}, 'caring': {'precision': 0.6341463414634146, 'recall': 0.1925925925925926, 'f1-score': 0.29545454545454547, 'support': 135.0}, 'confusion': {'precision': 0.64, 'recall': 0.20915032679738563, 'f1-score': 0.31527093596059114, 'support': 153.0}, 'curiosity': {'precision': 0.5093632958801498, 'recall': 0.4788732394366197, 'f1-score': 0.49364791288566245, 'support': 284.0}, 'desire': {'precision': 0.5925925925925926, 'recall': 0.1927710843373494, 'f1-score': 0.2909090909090909, 'support': 83.0}, 'disappointment': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 151.0}, 'disapproval': {'precision': 0.5806451612903226, 'recall': 0.20224719101123595, 'f1-score': 0.3, 'support': 267.0}, 'disgust': {'precision': 0.8235294117647058, 'recall': 0.11382113821138211, 'f1-score': 0.2, 'support': 123.0}, 'embarrassment': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 37.0}, 'excitement': {'precision': 0.6896551724137931, 'recall': 0.1941747572815534, 'f1-score': 0.30303030303030304, 'support': 103.0}, 'fear': {'precision': 0.6551724137931034, 'recall': 0.48717948717948717, 'f1-score': 0.5588235294117647, 'support': 78.0}, 'gratitude': {'precision': 0.9424242424242424, 'recall': 0.8835227272727273, 'f1-score': 0.9120234604105572, 'support': 352.0}, 'grief': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 6.0}, 'joy': {'precision': 0.675, 'recall': 0.5031055900621118, 'f1-score': 0.5765124555160143, 'support': 161.0}, 'love': {'precision': 0.7760617760617761, 'recall': 0.8445378151260504, 'f1-score': 0.8088531187122736, 'support': 238.0}, 'nervousness': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 23.0}, 'optimism': {'precision': 0.6929133858267716, 'recall': 0.4731182795698925, 'f1-score': 0.5623003194888179, 'support': 186.0}, 'pride': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 16.0}, 'realization': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 145.0}, 'relief': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 11.0}, 'remorse': {'precision': 0.6206896551724138, 'recall': 0.6428571428571429, 'f1-score': 0.631578947368421, 'support': 56.0}, 'sadness': {'precision': 0.6823529411764706, 'recall': 0.3717948717948718, 'f1-score': 0.48132780082987553, 'support': 156.0}, 'surprise': {'precision': 0.6470588235294118, 'recall': 0.3900709219858156, 'f1-score': 0.48672566371681414, 'support': 141.0}, 'neutral': {'precision': 0.734206471494607, 'recall': 0.5332960268606604, 'f1-score': 0.6178282009724473, 'support': 1787.0}, 'micro avg': {'precision': 0.7070608355729294, 'recall': 0.45726023068415234, 'f1-score': 0.5553636538092497, 'support': 6329.0}, 'macro avg': {'precision': 0.5017852603045123, 'recall': 0.32049624185387354, 'f1-score': 0.3657711628430001, 'support': 6329.0}, 'weighted avg': {'precision': 0.6478967641689403, 'recall': 0.45726023068415234, 'f1-score': 0.5120091314430566, 'support': 6329.0}, 'samples avg': {'precision': 0.5119464406363244, 'recall': 0.48252564338799836, 'f1-score': 0.4887660463116516, 'support': 6329.0}}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Exact Match Accuracy	Precision Micro	Recall Micro	F1 Micro	Precision Macro	Recall Macro	F1 Macro	Classification Report
0.1149	1.0	2714	0.0875	0.4342	0.7093	0.4672	0.5634	0.5096	0.3234	0.3691	{'admiration': {'precision': 0.706766917293233, 'recall': 0.7704918032786885, 'f1-score': 0.7372549019607844, 'support': 488.0}, 'amusement': {'precision': 0.7522123893805309, 'recall': 0.8415841584158416, 'f1-score': 0.794392523364486, 'support': 303.0}, 'anger': {'precision': 0.5093167701863354, 'recall': 0.4205128205128205, 'f1-score': 0.4606741573033708, 'support': 195.0}, 'annoyance': {'precision': 0.5897435897435898, 'recall': 0.07590759075907591, 'f1-score': 0.13450292397660818, 'support': 303.0}, 'approval': {'precision': 0.6013986013986014, 'recall': 0.21662468513853905, 'f1-score': 0.31851851851851853, 'support': 397.0}, 'caring': {'precision': 0.64, 'recall': 0.20915032679738563, 'f1-score': 0.31527093596059114, 'support': 153.0}, 'confusion': {'precision': 0.6444444444444445, 'recall': 0.19078947368421054, 'f1-score': 0.29441624365482233, 'support': 152.0}, 'curiosity': {'precision': 0.5495867768595041, 'recall': 0.5362903225806451, 'f1-score': 0.5428571428571428, 'support': 248.0}, 'desire': {'precision': 0.7857142857142857, 'recall': 0.2857142857142857, 'f1-score': 0.41904761904761906, 'support': 77.0}, 'disappointment': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 163.0}, 'disapproval': {'precision': 0.6891891891891891, 'recall': 0.17465753424657535, 'f1-score': 0.2786885245901639, 'support': 292.0}, 'disgust': {'precision': 0.6363636363636364, 'recall': 0.07216494845360824, 'f1-score': 0.12962962962962962, 'support': 97.0}, 'embarrassment': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 35.0}, 'excitement': {'precision': 0.625, 'recall': 0.15625, 'f1-score': 0.25, 'support': 96.0}, 'fear': {'precision': 0.8043478260869565, 'recall': 0.4111111111111111, 'f1-score': 0.5441176470588235, 'support': 90.0}, 'gratitude': {'precision': 0.9347181008902077, 'recall': 0.8798882681564246, 'f1-score': 0.9064748201438849, 'support': 358.0}, 'grief': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 13.0}, 'joy': {'precision': 0.717948717948718, 'recall': 0.4883720930232558, 'f1-score': 0.5813148788927336, 'support': 172.0}, 'love': {'precision': 0.706081081081081, 'recall': 0.8293650793650794, 'f1-score': 0.7627737226277372, 'support': 252.0}, 'nervousness': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 21.0}, 'optimism': {'precision': 0.7253521126760564, 'recall': 0.49282296650717705, 'f1-score': 0.5868945868945868, 'support': 209.0}, 'pride': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 15.0}, 'realization': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 127.0}, 'relief': {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 18.0}, 'remorse': {'precision': 0.7241379310344828, 'recall': 0.6176470588235294, 'f1-score': 0.6666666666666666, 'support': 68.0}, 'sadness': {'precision': 0.5471698113207547, 'recall': 0.40559440559440557, 'f1-score': 0.46586345381526106, 'support': 143.0}, 'surprise': {'precision': 0.6511627906976745, 'recall': 0.43410852713178294, 'f1-score': 0.5209302325581395, 'support': 129.0}, 'neutral': {'precision': 0.7279577995478523, 'recall': 0.5469988674971688, 'f1-score': 0.6246362754607178, 'support': 1766.0}, 'micro avg': {'precision': 0.709255293837735, 'recall': 0.4672413793103448, 'f1-score': 0.5633563261835018, 'support': 6380.0}, 'macro avg': {'precision': 0.5095933132806119, 'recall': 0.32343022595684323, 'f1-score': 0.3691044787493674, 'support': 6380.0}, 'weighted avg': {'precision': 0.6524027035593517, 'recall': 0.4672413793103448, 'f1-score': 0.5181766862517906, 'support': 6380.0}, 'samples avg': {'precision': 0.5241122988082073, 'recall': 0.4924130728590736, 'f1-score': 0.4997849858705, 'support': 6380.0}}

Framework versions

• Transformers 4.50.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1