roberta-large-emopillars-contextless

This model is a fine-tuned version of roberta-large on EmoPillars' context-less subset.

Model description

The model is a multi-label classifier over 28 emotional classes for a context-less scenario. It detects emotions in the entire input (including context if provided).

How to use

Here is how to use this model:

>>> import torch
>>> from transformers import pipeline
>>> model_name = "roberta-large-emopillars-contextless"
>>> threshold = 0.5
>>> emotions = [
>>>     "admiration", "amusement", "anger", "annoyance", "approval", "caring", "confusion",
>>>     "curiosity", "desire", "disappointment", "disapproval", "disgust", "embarrassment",
>>>     "excitement", "fear", "gratitude", "grief", "joy", "love", "nervousness", "optimism",
>>>     "pride", "realization", "relief", "remorse", "sadness", "surprise", "neutral"
>>> ]
>>> label_to_emotion = dict(zip(list(range(len(emotions))), emotions))
>>> device = torch.device("cuda" if torch.cuda.is_available() else "CPU")
>>> pipe = pipeline("text-classification", model=model_name, truncation=True,
>>>                  return_all_scores=True, device=-1 if device.type=="cpu" else 0)
>>> # input in a format f"{text}"
>>> utterances = [
>>>     "Ok is it just me or is anyone else getting goosebumps too???",
>>>     "Don’t know what to do"
>>> ]
>>> outcome = pipe(utterances)
>>> dominant_classes = [
>>>     [prediction for prediction in example if prediction['score'] >= threshold]
>>>     for example in outcome
>>> ]
>>> for example in dominant_classes:
>>>     print(", ".join([
>>>         "%s: %.2lf" % (label_to_emotion[int(prediction['label'])], prediction['score']) 
>>>         for prediction in sorted(example, key=lambda x: x['score'], reverse=True)
>>>     ]))
curiosity: 0.77, fear: 0.69, nervousness: 0.64
confusion: 1.00, nervousness: 1.00, annoyance: 0.85

Training data

The training data consists of 266,456 samples of EmoPillars' context-less subset created using Mistral within our data synthesis pipeline EmoPillars on GitHub. WikiPlots was used as a seed corpus.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 8
• seed: 752
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10.0

Framework versions

• Transformers 4.45.0.dev0
• Pytorch 2.4.0a0+gite3b9b71
• Datasets 2.21.0
• Tokenizers 0.19.1

Evaluation

Scores for the evaluation on the EmoPillars' test split:

class	precision	recall	f1-score	support
admiration	0.80	0.80	0.80	4113
amusement	0.83	0.75	0.79	1255
anger	0.90	0.90	0.90	7376
annoyance	0.87	0.85	0.86	5675
approval	0.62	0.60	0.61	1359
caring	0.76	0.75	0.75	2925
confusion	0.84	0.83	0.83	8257
curiosity	0.88	0.87	0.87	5067
desire	0.85	0.84	0.85	5411
disappointment	0.87	0.87	0.87	8118
disapproval	0.70	0.66	0.68	2883
disgust	0.82	0.80	0.81	1901
embarrassment	0.78	0.68	0.73	719
excitement	0.80	0.79	0.80	4532
fear	0.91	0.91	0.91	6029
gratitude	0.85	0.82	0.84	989
grief	0.77	0.75	0.76	944
joy	0.82	0.84	0.83	4100
love	0.80	0.80	0.80	1725
nervousness	0.86	0.85	0.86	8344
optimism	0.87	0.85	0.86	3214
pride	0.90	0.89	0.89	3159
realization	0.79	0.64	0.70	635
relief	0.81	0.80	0.80	1325
remorse	0.72	0.66	0.69	1297
sadness	0.85	0.89	0.87	5744
surprise	0.87	0.86	0.86	5527
neutral	0.78	0.70	0.74	2869
micro avg	0.84	0.83	0.84	105492
macro avg	0.82	0.79	0.81	105492
weighted avg	0.84	0.83	0.84	105492
samples avg	0.85	0.84	0.83	105492

When fine-tuned on downstream tasks, this model achieves the following results:

task	precision	recall	f1-score
GoEmotions	0.53	0.58	0.55
ISEAR	0.76	0.75	0.75

For more details on the evaluation, please visit our GitHub repository or paper.

Citation information

If you use this model, please cite our paper:

@misc{shvets2025emopillarsknowledgedistillation,
      title={Emo Pillars: Knowledge Distillation to Support Fine-Grained Context-Aware and Context-Less Emotion Classification}, 
      author={Alexander Shvets},
      year={2025},
      eprint={2504.16856},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2504.16856}
}

Disclaimer

Click to expand

The model published in this repository is intended for a generalist purpose and is available to third parties. This model may have bias and/or any other undesirable distortions.

When third parties deploy or provide systems and/or services to other parties using this model (or using systems based on this model) or become users of the model, they should note that it is their responsibility to mitigate the risks arising from its use and, in any event, to comply with applicable regulations, including regulations regarding the use of Artificial Intelligence.

In no event shall the creator of the model be liable for any results arising from the use made by third parties of this model.