Hugging Face's logo

nbeerbower
/
Qwen2.5-Gutenberg-Doppel-14B

Text Generation
Transformers
Safetensors
qwen2
conversational
Eval Results
text-generation-inference
Model card Files Community
2
nbeerbower lbourdois commited on
Commit
788b062
·
verified ·
1 Parent(s): 27f41b2

Improve language tag (#2)

Browse files

- Improve language tag (d0b38753b2a03d9ed8bc68bfc4002e0c6f6a81fa)


Co-authored-by: Loïck BOURDOIS <[email protected]>

Files changed (1) hide show
  1. README.md +143 -129
README.md CHANGED
@@ -1,129 +1,143 @@
1
- ---
2
- license: apache-2.0
3
- library_name: transformers
4
- base_model:
5
- - Qwen/Qwen2.5-14B-Instruct
6
- datasets:
7
- - jondurbin/gutenberg-dpo-v0.1
8
- - nbeerbower/gutenberg2-dpo
9
- model-index:
10
- - name: Qwen2.5-Gutenberg-Doppel-14B
11
- results:
12
- - task:
13
- type: text-generation
14
- name: Text Generation
15
- dataset:
16
- name: IFEval (0-Shot)
17
- type: HuggingFaceH4/ifeval
18
- args:
19
- num_few_shot: 0
20
- metrics:
21
- - type: inst_level_strict_acc and prompt_level_strict_acc
22
- value: 80.91
23
- name: strict accuracy
24
- source:
25
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=nbeerbower/Qwen2.5-Gutenberg-Doppel-14B
26
- name: Open LLM Leaderboard
27
- - task:
28
- type: text-generation
29
- name: Text Generation
30
- dataset:
31
- name: BBH (3-Shot)
32
- type: BBH
33
- args:
34
- num_few_shot: 3
35
- metrics:
36
- - type: acc_norm
37
- value: 48.24
38
- name: normalized accuracy
39
- source:
40
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=nbeerbower/Qwen2.5-Gutenberg-Doppel-14B
41
- name: Open LLM Leaderboard
42
- - task:
43
- type: text-generation
44
- name: Text Generation
45
- dataset:
46
- name: MATH Lvl 5 (4-Shot)
47
- type: hendrycks/competition_math
48
- args:
49
- num_few_shot: 4
50
- metrics:
51
- - type: exact_match
52
- value: 0.0
53
- name: exact match
54
- source:
55
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=nbeerbower/Qwen2.5-Gutenberg-Doppel-14B
56
- name: Open LLM Leaderboard
57
- - task:
58
- type: text-generation
59
- name: Text Generation
60
- dataset:
61
- name: GPQA (0-shot)
62
- type: Idavidrein/gpqa
63
- args:
64
- num_few_shot: 0
65
- metrics:
66
- - type: acc_norm
67
- value: 11.07
68
- name: acc_norm
69
- source:
70
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=nbeerbower/Qwen2.5-Gutenberg-Doppel-14B
71
- name: Open LLM Leaderboard
72
- - task:
73
- type: text-generation
74
- name: Text Generation
75
- dataset:
76
- name: MuSR (0-shot)
77
- type: TAUR-Lab/MuSR
78
- args:
79
- num_few_shot: 0
80
- metrics:
81
- - type: acc_norm
82
- value: 10.02
83
- name: acc_norm
84
- source:
85
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=nbeerbower/Qwen2.5-Gutenberg-Doppel-14B
86
- name: Open LLM Leaderboard
87
- - task:
88
- type: text-generation
89
- name: Text Generation
90
- dataset:
91
- name: MMLU-PRO (5-shot)
92
- type: TIGER-Lab/MMLU-Pro
93
- config: main
94
- split: test
95
- args:
96
- num_few_shot: 5
97
- metrics:
98
- - type: acc
99
- value: 43.57
100
- name: accuracy
101
- source:
102
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=nbeerbower/Qwen2.5-Gutenberg-Doppel-14B
103
- name: Open LLM Leaderboard
104
- ---
105
-
106
- ![image/png](https://huggingface.co/nbeerbower/Mistral-Small-Gutenberg-Doppel-22B/resolve/main/doppel-header?download=true)
107
-
108
- # Qwen2.5-Gutenberg-Doppel-14B
109
-
110
- [Qwen/Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct) finetuned on [jondurbin/gutenberg-dpo-v0.1](https://huggingface.co/datasets/jondurbin/gutenberg-dpo-v0.1) and [nbeerbower/gutenberg2-dpo](https://huggingface.co/datasets/nbeerbower/gutenberg2-dpo).
111
-
112
- ### Method
113
-
114
- [ORPO tuned](https://mlabonne.github.io/blog/posts/2024-04-19_Fine_tune_Llama_3_with_ORPO.html) with 4x A40 for 3 epochs.
115
-
116
- Thank you [@ParasiticRogue](https://huggingface.co/ParasiticRogue) for sponsoring.
117
- # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
118
- Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_nbeerbower__Qwen2.5-Gutenberg-Doppel-14B)
119
-
120
- | Metric |Value|
121
- |-------------------|----:|
122
- |Avg. |32.30|
123
- |IFEval (0-Shot) |80.91|
124
- |BBH (3-Shot) |48.24|
125
- |MATH Lvl 5 (4-Shot)| 0.00|
126
- |GPQA (0-shot) |11.07|
127
- |MuSR (0-shot) |10.02|
128
- |MMLU-PRO (5-shot) |43.57|
129
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: transformers
4
+ base_model:
5
+ - Qwen/Qwen2.5-14B-Instruct
6
+ datasets:
7
+ - jondurbin/gutenberg-dpo-v0.1
8
+ - nbeerbower/gutenberg2-dpo
9
+ language:
10
+ - zho
11
+ - eng
12
+ - fra
13
+ - spa
14
+ - por
15
+ - deu
16
+ - ita
17
+ - rus
18
+ - jpn
19
+ - kor
20
+ - vie
21
+ - tha
22
+ - ara
23
+ model-index:
24
+ - name: Qwen2.5-Gutenberg-Doppel-14B
25
+ results:
26
+ - task:
27
+ type: text-generation
28
+ name: Text Generation
29
+ dataset:
30
+ name: IFEval (0-Shot)
31
+ type: HuggingFaceH4/ifeval
32
+ args:
33
+ num_few_shot: 0
34
+ metrics:
35
+ - type: inst_level_strict_acc and prompt_level_strict_acc
36
+ value: 80.91
37
+ name: strict accuracy
38
+ source:
39
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=nbeerbower/Qwen2.5-Gutenberg-Doppel-14B
40
+ name: Open LLM Leaderboard
41
+ - task:
42
+ type: text-generation
43
+ name: Text Generation
44
+ dataset:
45
+ name: BBH (3-Shot)
46
+ type: BBH
47
+ args:
48
+ num_few_shot: 3
49
+ metrics:
50
+ - type: acc_norm
51
+ value: 48.24
52
+ name: normalized accuracy
53
+ source:
54
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=nbeerbower/Qwen2.5-Gutenberg-Doppel-14B
55
+ name: Open LLM Leaderboard
56
+ - task:
57
+ type: text-generation
58
+ name: Text Generation
59
+ dataset:
60
+ name: MATH Lvl 5 (4-Shot)
61
+ type: hendrycks/competition_math
62
+ args:
63
+ num_few_shot: 4
64
+ metrics:
65
+ - type: exact_match
66
+ value: 0.0
67
+ name: exact match
68
+ source:
69
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=nbeerbower/Qwen2.5-Gutenberg-Doppel-14B
70
+ name: Open LLM Leaderboard
71
+ - task:
72
+ type: text-generation
73
+ name: Text Generation
74
+ dataset:
75
+ name: GPQA (0-shot)
76
+ type: Idavidrein/gpqa
77
+ args:
78
+ num_few_shot: 0
79
+ metrics:
80
+ - type: acc_norm
81
+ value: 11.07
82
+ name: acc_norm
83
+ source:
84
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=nbeerbower/Qwen2.5-Gutenberg-Doppel-14B
85
+ name: Open LLM Leaderboard
86
+ - task:
87
+ type: text-generation
88
+ name: Text Generation
89
+ dataset:
90
+ name: MuSR (0-shot)
91
+ type: TAUR-Lab/MuSR
92
+ args:
93
+ num_few_shot: 0
94
+ metrics:
95
+ - type: acc_norm
96
+ value: 10.02
97
+ name: acc_norm
98
+ source:
99
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=nbeerbower/Qwen2.5-Gutenberg-Doppel-14B
100
+ name: Open LLM Leaderboard
101
+ - task:
102
+ type: text-generation
103
+ name: Text Generation
104
+ dataset:
105
+ name: MMLU-PRO (5-shot)
106
+ type: TIGER-Lab/MMLU-Pro
107
+ config: main
108
+ split: test
109
+ args:
110
+ num_few_shot: 5
111
+ metrics:
112
+ - type: acc
113
+ value: 43.57
114
+ name: accuracy
115
+ source:
116
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=nbeerbower/Qwen2.5-Gutenberg-Doppel-14B
117
+ name: Open LLM Leaderboard
118
+ ---
119
+
120
+ ![image/png](https://huggingface.co/nbeerbower/Mistral-Small-Gutenberg-Doppel-22B/resolve/main/doppel-header?download=true)
121
+
122
+ # Qwen2.5-Gutenberg-Doppel-14B
123
+
124
+ [Qwen/Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct) finetuned on [jondurbin/gutenberg-dpo-v0.1](https://huggingface.co/datasets/jondurbin/gutenberg-dpo-v0.1) and [nbeerbower/gutenberg2-dpo](https://huggingface.co/datasets/nbeerbower/gutenberg2-dpo).
125
+
126
+ ### Method
127
+
128
+ [ORPO tuned](https://mlabonne.github.io/blog/posts/2024-04-19_Fine_tune_Llama_3_with_ORPO.html) with 4x A40 for 3 epochs.
129
+
130
+ Thank you [@ParasiticRogue](https://huggingface.co/ParasiticRogue) for sponsoring.
131
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
132
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_nbeerbower__Qwen2.5-Gutenberg-Doppel-14B)
133
+
134
+ | Metric |Value|
135
+ |-------------------|----:|
136
+ |Avg. |32.30|
137
+ |IFEval (0-Shot) |80.91|
138
+ |BBH (3-Shot) |48.24|
139
+ |MATH Lvl 5 (4-Shot)| 0.00|
140
+ |GPQA (0-shot) |11.07|
141
+ |MuSR (0-shot) |10.02|
142
+ |MMLU-PRO (5-shot) |43.57|
143
+