Probably DPO datasets

davanstrien 's Collections

Reasoning Required?

hub-tldr

Maths reasoning

synthetic-data-generation-demos

sentence-transformers-from-synthetic-data

Synthetic (text) Dataset Generation

haiku

Historic language modeling

Probably DPO datasets

Image Preference Optimization Datasets

query-to-hub-datasets-viewer-project

updated Jun 26, 2024

A collection of datasets that probably support DPO

Upvote

HuggingFaceH4/ultrafeedback_binarized

Viewer • Updated Oct 16, 2024 • 187k • 8.12k • 290
mlabonne/orpo-dpo-mix-40k

Viewer • Updated Oct 17, 2024 • 44.2k • 1.94k • 281
argilla/OpenHermesPreferences

Viewer • Updated Mar 1, 2024 • 989k • 508 • 204
argilla/distilabel-capybara-dpo-7k-binarized

Viewer • Updated Jul 16, 2024 • 7.56k • 1.64k • 180
argilla/distilabel-intel-orca-dpo-pairs

Viewer • Updated Mar 19 • 12.9k • 1.9k • 173
argilla/dpo-mix-7k

Viewer • Updated Jul 16, 2024 • 7.5k • 572 • 162
argilla/ultrafeedback-binarized-preferences-cleaned

Viewer • Updated Dec 11, 2023 • 60.9k • 1.05k • 142
Dahoas/rm-static

Viewer • Updated Mar 6, 2023 • 81.4k • 3.63k • 120
openbmb/UltraInteract_pair

Viewer • Updated Apr 5, 2024 • 220k • 880 • 107
openbmb/RLAIF-V-Dataset

Viewer • Updated Mar 4 • 74.8k • 1.56k • 171
llamafactory/DPO-En-Zh-20k

Viewer • Updated Jun 7, 2024 • 20k • 394 • 91
Dahoas/full-hh-rlhf

Viewer • Updated Feb 23, 2023 • 125k • 3.04k • 81
argilla/OpenHermes2.5-dpo-binarized-alpha

Viewer • Updated Feb 10, 2024 • 9.79k • 26 • 63
yitingxie/rlhf-reward-datasets

Viewer • Updated Jan 1, 2023 • 81.4k • 270 • 65
allenai/ultrafeedback_binarized_cleaned

Viewer • Updated Dec 1, 2023 • 186k • 944 • 69
allenai/reward-bench

Viewer • Updated Sep 9, 2024 • 8.11k • 7.59k • 94
CarperAI/openai_summarize_comparisons

Viewer • Updated Feb 27, 2023 • 260k • 3.4k • 40
beyond/rlhf-reward-single-round-trans_chinese

Viewer • Updated Jul 5, 2023 • 24.9k • 104 • 39
argilla/Capybara-Preferences

Viewer • Updated May 9, 2024 • 15.4k • 124 • 42
snorkelai/Snorkel-Mistral-PairRM-DPO-Dataset

Viewer • Updated Jan 23, 2024 • 62.7k • 167 • 43
tasksource/oasst1_pairwise_rlhf_reward

Viewer • Updated Jul 4, 2023 • 18.9k • 189 • 43
wenbopan/Chinese-dpo-pairs

Viewer • Updated Apr 2, 2024 • 10.7k • 72 • 44
maywell/ko_Ultrafeedback_binarized

Viewer • Updated Nov 9, 2023 • 62k • 163 • 31
abacusai/MetaMath_DPO_FewShot

Viewer • Updated Feb 26, 2024 • 395k • 114 • 26
HuggingFaceH4/orca_dpo_pairs

Viewer • Updated Apr 14, 2024 • 12.9k • 769 • 29
Dahoas/static-hh

Viewer • Updated Mar 6, 2023 • 101k • 67 • 21
mlabonne/chatml-OpenHermes2.5-dpo-binarized-alpha

Viewer • Updated Mar 21, 2024 • 9.2k • 38 • 19
HuggingFaceH4/grok-conversation-harmless

Viewer • Updated Feb 2, 2024 • 44.8k • 255 • 25
allenai/preference-test-sets

Viewer • Updated Mar 14, 2024 • 43.2k • 1.08k • 25
abacusai/ARC_DPO_FewShot

Viewer • Updated Feb 26, 2024 • 4.25k • 50 • 18
cyberagent/chatbot-arena-ja-calm2-7b-chat-experimental

Viewer • Updated Aug 15, 2024 • 29.2k • 207 • 19
vicgalle/configurable-system-prompt-multitask

Viewer • Updated Apr 23, 2024 • 1.95k • 134 • 26
coseal/CodeUltraFeedback_binarized

Viewer • Updated Mar 18, 2024 • 9.5k • 71 • 17
NobodyExistsOnTheInternet/ToxicDPOqa

Viewer • Updated Apr 26, 2024 • 6.87k • 71 • 17
multilingual/orca_dpo_pairs

Viewer • Updated Apr 3, 2024 • 64.7k • 110 • 18
fblgit/simple-math-DPO

Viewer • Updated Aug 1, 2024 • 800k • 88 • 17
HuggingFaceH4/cai-conversation-harmless

Viewer • Updated Feb 2, 2024 • 44.8k • 268 • 15
alvarobartt/dpo-mix-7k-simplified

Viewer • Updated Mar 26, 2024 • 7.5k • 32 • 11
kyujinpy/orca_math_dpo

Viewer • Updated Apr 12, 2024 • 15.3k • 55 • 19
davanstrien/dataset-tldr-preference-dpo

Viewer • Updated Apr 30, 2024 • 522 • 90 • 11
trl-internal-testing/hh-rlhf-trl-style

Viewer • Updated Mar 13, 2024 • 169k • 124 • 11
abacusai/HellaSwag_DPO_FewShot

Viewer • Updated Feb 26, 2024 • 150k • 43 • 8
2A2I/argilla-dpo-mix-7k-arabic

Viewer • Updated Mar 18, 2024 • 7.5k • 85 • 10
argilla/Capybara-Preferences-Filtered

Viewer • Updated Apr 20, 2024 • 14.8k • 45 • 9
2A2I/Aya-Command.R-DPO

Viewer • Updated May 16, 2024 • 14.2k • 50 • 7
kuotient/orca-math-korean-preference

Viewer • Updated Apr 10, 2024 • 193k • 159 • 10
M4-ai/prm_dpo_pairs

Viewer • Updated Jul 1, 2024 • 93.9k • 77 • 7
zerolink/zsql-postgres-dpo

Viewer • Updated Feb 2, 2024 • 259k • 180 • 8
princeton-nlp/llama3-ultrafeedback

Viewer • Updated Jul 18, 2024 • 61.8k • 851 • 18
kaist-ai/Multifaceted-Collection-ORPO

Viewer • Updated Jul 1, 2024 • 64.6k • 63 • 10
kaitchup/UltraFeedback-prompt-chosen-rejected

Viewer • Updated Oct 23, 2023 • 18k • 78 • 6
alexredna/oasst2_dpo_pairs

Viewer • Updated Jul 17, 2024 • 10k • 43 • 6
HuggingFaceH4/grok-conversation-harmless2

Viewer • Updated Jan 19, 2024 • 44.8k • 87 • 7
yongtae-jp/orca_dpo_pairs_ja

Viewer • Updated Jan 9, 2024 • 12.9k • 39 • 6
kaist-ai/Multifaceted-Collection-DPO

Viewer • Updated Jul 1, 2024 • 65.1k • 54 • 9
malhajar/distilabel-intel-orca-dpo-pairs-tr

Viewer • Updated Mar 28, 2024 • 9.12k • 29 • 6
efederici/alpaca-vs-alpaca-orpo-dpo

Viewer • Updated May 15, 2024 • 49.2k • 53 • 7
maywell/ko_Ultrafeedback_binarized_10k

Viewer • Updated Nov 4, 2023 • 10k • 53 • 5
kira/math-dpo

Viewer • Updated Jul 2, 2024 • 2.42k • 242 • 8
mlabonne/distilabel-truthy-dpo-v0.1

Viewer • Updated Feb 12, 2024 • 1.02k • 17 • 5
RLHFlow/UltraFeedback-preference-standard

Viewer • Updated Apr 27, 2024 • 340k • 82 • 12
BramVanroy/orca_dpo_pairs_dutch

Viewer • Updated Dec 6, 2024 • 11k • 69 • 6
argilla/ultrafeedback-multi-binarized-preferences-cleaned

Viewer • Updated Dec 11, 2023 • 158k • 51 • 7
malhajar/orca_dpo_pairs-tr

Viewer • Updated Mar 30, 2024 • 10.8k • 28 • 7
lightblue/mitsu_tophalf_borda

Viewer • Updated May 30, 2024 • 1.42k • 81 • 5
NickyNicky/neovalle_H4rmony_dpo_translated_English_to_Spanish

Viewer • Updated May 17, 2024 • 2.02k • 16 • 5
maywell/ko_hh-rlhf-20k_filtered

Viewer • Updated Nov 4, 2023 • 19.4k • 29 • 4
kaist-ai/Multifaceted-Collection-RM

Viewer • Updated Jun 7, 2024 • 64.9k • 109 • 4
HuggingFaceH4/hh-rlhf-h4

Viewer • Updated Nov 29, 2023 • 169k • 28 • 6
orpo-explorers/ultrafeedback_binarized_15k

Viewer • Updated Apr 18, 2024 • 15.4k • 27 • 4
We-Want-GPU/Yi-Ko-DPO-Orca-DPO-Pairs

Viewer • Updated Dec 18, 2023 • 12.7k • 22 • 5
zerolink/zsql-sqlite-dpo

Viewer • Updated Feb 2, 2024 • 260k • 105 • 3
argilla/ultrafeedback-multi-binarized-quality-preferences-cleaned

Viewer • Updated Dec 11, 2023 • 155k • 41 • 4
SAGI-1/Greetings_DPO_dataset_V1

Viewer • Updated Dec 18, 2023 • 891 • 28 • 5
2A2I/Aya-SambaLingo.Arabic.Chat-DPO

Viewer • Updated May 25, 2024 • 14.2k • 77 • 3
AlekseyKorshuk/evol-codealpaca-v1-dpo

Viewer • Updated Jan 25, 2024 • 39.9k • 67 • 6
mlabonne/ultrafeedback-binarized-preferences-cleaned

Viewer • Updated Mar 27, 2024 • 28k • 38 • 3
kuotient/orca-math-korean-dpo-pairs

Viewer • Updated Apr 5, 2024 • 193k • 99 • 11
mii-community/ultrafeedback-preferences-translated-ita

Viewer • Updated Feb 21, 2024 • 60.9k • 35 • 3
sqrti/SPA-VL

Viewer • Updated Jul 3, 2024 • 101k • 738 • 8
HuggingFaceH4/orca_dpo_pairs_no_system_prompt

Viewer • Updated Jan 15, 2024 • 12.9k • 13 • 4
mlabonne/distilabel-intel-orca-dpo-pairs

Viewer • Updated Feb 6, 2024 • 5.92k • 18 • 3
d0rj/rlhf-reward-datasets-ru

Viewer • Updated Jul 2, 2024 • 81.4k • 94 • 4
RLHFlow/HH-RLHF-Harmless-and-RedTeam-standard

Viewer • Updated May 8, 2024 • 42.3k • 53 • 3
2A2I/Aya-AceGPT.13B.Chat-DPO

Viewer • Updated May 16, 2024 • 13k • 54 • 2
RLHFlow/Helpsteer-preference-standard

Viewer • Updated Apr 27, 2024 • 37.1k • 33 • 6
IlyaGusev/saiga_preferences

Viewer • Updated Oct 28, 2024 • 30.6k • 76 • 7
RLHFlow/CodeUltraFeedback-standard

Viewer • Updated Apr 27, 2024 • 50.2k • 48 • 5
RLHFlow/UltraInteract-filtered-standard

Viewer • Updated Apr 28, 2024 • 162k • 19 • 2
RLHFlow/Prometheus2-preference-standard

Viewer • Updated May 5, 2024 • 200k • 41 • 2
RLHFlow/PKU-SafeRLHF-30K-standard

Viewer • Updated Apr 29, 2024 • 26.9k • 27 • 3
RLHFlow/Argilla-Math-DPO-standard

Viewer • Updated Apr 30, 2024 • 2.42k • 21 • 3
droussis/UltraSafety_binarized-orpo-dpo

Viewer • Updated Apr 25, 2024 • 2.26k • 20 • 2
We-Want-GPU/Yi-Ko-DPO-dataset

Viewer • Updated Dec 9, 2023 • 4k • 16 • 4
roborovski/glaive-tool-usage-dpo

Viewer • Updated Feb 29, 2024 • 42k • 119 • 2
vicgalle/OpenHermesPreferences-roleplay

Viewer • Updated Feb 29, 2024 • 3.06k • 124 • 8
GrowingApple/orpo_kor_translated_medical

Viewer • Updated Jun 12, 2024 • 10.4k • 10 • 3
reciprocate/ultrafeedback_cleaned_high_dpo

Viewer • Updated Jan 11, 2024 • 40k • 18 • 2
mlabonne/distilabel-truthy-dpo-v0.1-filtered

Viewer • Updated Feb 15, 2024 • 663 • 14 • 2
xiaodongguaAIGC/awesome-dpo

Viewer • Updated Jul 3, 2024 • 302k • 35 • 3
jan-hq/evol_codealpaca_dpo_binarized

Viewer • Updated Feb 20, 2024 • 39.9k • 49 • 2
2A2I/Aya-Aya.23.8B-DPO

Viewer • Updated May 29, 2024 • 14k • 26 • 2
tasksource/oasst2_pairwise_rlhf_reward

Viewer • Updated Jan 9, 2024 • 28.4k • 28 • 2
NickyNicky/function_call_orpo_sft_phi3_chatML_only

Viewer • Updated May 2, 2024 • 66k • 20 • 2
HuggingFaceH4/summarize_from_feedback

Viewer • Updated Mar 11, 2024 • 353k • 45 • 3
weqweasdas/preference_dataset_mix2

Viewer • Updated Mar 21, 2024 • 528k • 48 • 3
NickyNicky/oasst2_orpo_mix_function_call_phi_3_v1

Viewer • Updated May 2, 2024 • 120k • 14 • 2
BramVanroy/ultra_feedback_dutch

Viewer • Updated Dec 6, 2024 • 53.6k • 117 • 2
orpo-explorers/OpenHermesPreferences-10k

Viewer • Updated Mar 30, 2024 • 10k • 7 • 2
orpo-explorers/OpenHermesPreferences-500k

Viewer • Updated Mar 30, 2024 • 500k • 54 • 2
zhengr/ultrafeedback_binarized

Viewer • Updated Nov 8, 2023 • 190k • 30 • 2
ibivibiv/cleaned_orca_math_dpo_pairs

Viewer • Updated May 24, 2024 • 200k • 17 • 2
NickyNicky/nano_finance_200k_en_es_chatML_gemma_orpo_dpo

Viewer • Updated May 29, 2024 • 201k • 8 • 1
Baidicoot/hh-rlhf-golden-harmful

Viewer • Updated May 10, 2024 • 7.64k • 38 • 1
Tachi67/rm_data_motivation

Viewer • Updated Jun 4, 2024 • 44.8k • 43 • 1
trl-internal-testing/hh-rlhf-helpful-base-trl-style

Viewer • Updated May 2, 2024 • 46.2k • 5.71k • 10
flyingfishinwater/ultrafeedback_clean

Viewer • Updated Jul 2, 2024 • 175k • 53 • 1
manishiitg/argilla-distilabel-intel-orca-dpo-pairs

Viewer • Updated Jan 29, 2024 • 23.6k • 19 • 1
Palash123/dpo_anthropic_hh_rlhf

Viewer • Updated Dec 4, 2023 • 168k • 40 • 1
David-Xu/astronomy-stack-dpo-text

Viewer • Updated Mar 8, 2024 • 19.9k • 59 • 2
d0rj/full-hh-rlhf-ru

Viewer • Updated Jul 2, 2024 • 125k • 35 • 2
vwxyzjn/summarize_from_feedback_oai_preprocessing_1706381144

Viewer • Updated Jan 27, 2024 • 179k • 130 • 2
reciprocate/ultrafeedback_orca_math_cleaned_high_dpo

Viewer • Updated Jan 11, 2024 • 48.3k • 16 • 1
HuggingFaceH4/OpenHermes-2.5-preferences-v0-deduped

Viewer • Updated Feb 9, 2024 • 802k • 50 • 1
mlabonne/truthy-dpo-v0.1

Viewer • Updated Feb 18, 2024 • 1.02k • 22 • 1
macadeliccc/distilabel-neurology-preferences-2k-orca-format

Viewer • Updated Feb 22, 2024 • 1.99k • 15 • 1
jan-hq/athirdpath_roleplay_dpo_binarized

Viewer • Updated Jan 9, 2024 • 3.43k • 71 • 1
reciprocate/ultrafeedback_orca_math_dpo

Viewer • Updated Jan 8, 2024 • 73.8k • 23 • 2
d0rj/synthetic-instruct-gptj-pairwise-ru

Viewer • Updated Jul 2, 2024 • 33.1k • 27 • 1
CorticalStack/tak-stack-dpo

Viewer • Updated Feb 28, 2024 • 10.9k • 18 • 1
umarigan/openhermespreference_tr

Viewer • Updated Apr 24, 2024 • 989k • 40 • 3
amztheory/toxic-aira-dataset

Viewer • Updated Apr 30, 2024 • 8.29k • 34 • 2
RLHFlow/HH-RLHF-Helpful-standard

Viewer • Updated Apr 27, 2024 • 115k • 208 • 1
trl-internal-testing/descriptiveness-sentiment-trl-style

Viewer • Updated Apr 9, 2024 • 10.9k • 1.5k • 1
We-Want-GPU/yi-ko-DPO-noprompt-dataset

Viewer • Updated Dec 6, 2023 • 3.83k • 20 • 2
hendrydong/preference_700K

Viewer • Updated Sep 28, 2024 • 700k • 317 • 15
insub/imdb_prefix20_forDPO_gpt2-large-imdb-FT_siebert_sentiment-roberta-large-english

Viewer • Updated Oct 22, 2023 • 50k • 28 • 2
FredrikBL/Legal-Snigel-DPO

Viewer • Updated May 27, 2024 • 2k • 21 • 1
lightblue/mitsu_full_borda

Viewer • Updated May 30, 2024 • 2.85k • 21 • 2
MaziyarPanahi/truthy-dpo-v0.1-axolotl

Viewer • Updated May 2, 2024 • 1.02k • 59 • 8
RLHFlow/Orca-distibalel-standard

Viewer • Updated Apr 28, 2024 • 6.93k • 17 • 1
RLHFlow/prm80k-phase2

Viewer • Updated Apr 28, 2024 • 79.5k • 21 • 4
pythainlp/oasst2_dpo_pairs_enth

Viewer • Updated Jun 16, 2024 • 5.28k • 54 • 1

Upvote

Collection guide
Browse collections