datasets:
- hamishivi/rds-sels-arena-hard-top326k
language:
- en
base_model:
- meta-llama/Llama-2-7b-hf
RDS+ Arena Hard Tulu 2 326k
This is a model trained on 326k samples selected by RDS+ using Arena Hard samples from the Tulu 2 unfiltered dataset. For more details, please see the paper Practical Large-Scale Data Selection for Instruction Tuning and associated codebase.
.Model description
- Model type: A model instruction-tuned on data selected from Tulu 2 unfiltered.
- Language(s) (NLP): English
- License: Llama 2 models are licensed under the Llama 2 license. A copy of this and a notice file can be found in this repository.
- Finetuned from model: meta-llama/Llama-2-7b-hf
Model Sources
- Repository: https://github.com/hamishivi/automated-instruction-selection
- Dataset: Data used to train this model can be found here.
- Model Family: The collection of related models can be found here.
Results
For more results and analysis, please see our paper.
| Method | MMLU | GSM8k | BBH | TydiQA | Codex | Squad | AlpacaEval | Average |
|---|---|---|---|---|---|---|---|---|
| Rand. (unbal) | 52.2 | 18.0 | 44.5 | 55.3 | 25.7 | 81.5 | 33.9 | 44.5 |
| Rand. (bal) | 51.5 | 21.8 | 45.1 | 50.7 | 32.2 | 87.9 | 43.2 | 47.5 |
| Top-PPL | 49.1 | 10.5 | 39.4 | 49.4 | 21.6 | 80.3 | 5.6 | 36.6 |
| Mid-PPL | 53.1 | 13.3 | 42.8 | 52.4 | 20.3 | 86.2 | 20.7 | 41.3 |
| Embed (GTR) | 49.9 | 32.8 | 44.6 | 54.4 | 30.4 | 88.4 | 35.7 | 48.0 |
| Embed (NV) | 50.6 | 28.7 | 44.4 | 56.0 | 30.4 | 89.1 | 17.9 | 45.3 |
| IFD | 45.7 | 21.8 | 41.2 | 39.5 | 27.7 | 17.0 | 57.4 | 35.7 |
| Tulu 2 | 50.0 | 22.7 | 45.1 | 54.0 | 33.1 | 86.9 | 54.4 | 49.5 |
| RDS+ (this model) | 50.2 | 35.2 | 44.7 | 56.3 | 35.1 | 89.0 | 45.6 | 50.9 |
| RDS+ - Wildchat | 50.9 | 24.8 | 43.6 | 57.3 | 31.1 | 87.3 | 39.3 | 47.8 |
| RDS+ - Arena Hard (this model) | 48.1 | 36.2 | 43.9 | 51.8 | 31.8 | 81.3 | 59.4 | 50.4 |
Input Format
The model is trained to use the following format (note the newlines):
<|user|>
Your message here!
<|assistant|>
For best results, format all inputs in this manner. Make sure to include a newline after <|assistant|>, this can affect generation quality quite a bit.
We have included a chat template in the tokenizer implementing this template.
Bias, Risks, and Limitations
These models have not been aligned to generate safe completions, so the model can produce problematic outputs (especially when prompted to do so).
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- total_train_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.03
- num_epochs: 2.0
Citation
If you find this model or data is useful in your work, please cite it with:
@misc{ivison2025data,
title={{Practical Large-Scale Data Selection for Instruction Tuning}},
author={{Hamish Ivison and Muru Zhang and Faeze Brahman and Pang Wei Koh and Pradeep Dasigi}}
year={2025},
url={https://arxiv.org/abs/2503.01807},
eprint={2503.01807},
archivePrefix={arXiv},
primaryClass={cs.CL}
}