Dongfu Jiang
commited on
Commit
·
adad0ff
1
Parent(s):
f5ed554
Update README.md
Browse files
README.md
CHANGED
@@ -18,15 +18,20 @@ tags:
|
|
18 |
PairRanker used in llm-blender, trained on deberta-v3-large. This is the ranker model used in experiments in LLM-Blender paper,
|
19 |
which is trained on [mixinstruct](https://huggingface.co/datasets/llm-blender/mix-instruct) dataset for 5 epochs.
|
20 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
21 |
| PairRanker type | Source max length | Candidate max length | Total max length |
|
22 |
|:-----------------:|:-----------------:|----------------------|------------------|
|
23 |
| [pair-ranker](https://huggingface.co/llm-blender/pair-ranker) (This model) | 128 | 128 | 384 |
|
24 |
| [pair-reward-model](https://huggingface.co/llm-blender/pair-reward-model/) | 1224 | 412 | 2048 |
|
25 |
|
26 |
|
27 |
-
|
28 |
-
- Paper: [https://arxiv.org/abs/2306.02561](https://arxiv.org/abs/2306.02561)
|
29 |
-
|
30 |
|
31 |
| **Methods** | BERTScore | BARTScore | BLEURT | GPT-Rank | Beat Vic(%) | Beat OA(%) | Top-1(%) | Top-2(%) | Top-3(%) |
|
32 |
|:-----------------:|:---------:|:---------:|:---------:|:--------:|:----------:|:----------:|:----------:|:----------:|:----------:|
|
|
|
18 |
PairRanker used in llm-blender, trained on deberta-v3-large. This is the ranker model used in experiments in LLM-Blender paper,
|
19 |
which is trained on [mixinstruct](https://huggingface.co/datasets/llm-blender/mix-instruct) dataset for 5 epochs.
|
20 |
|
21 |
+
- Github: [https://github.com/yuchenlin/LLM-Blender](https://github.com/yuchenlin/LLM-Blender)
|
22 |
+
- Paper: [https://arxiv.org/abs/2306.02561](https://arxiv.org/abs/2306.02561)
|
23 |
+
|
24 |
+
|
25 |
+
## Statistics
|
26 |
+
|
27 |
+
### Context length
|
28 |
| PairRanker type | Source max length | Candidate max length | Total max length |
|
29 |
|:-----------------:|:-----------------:|----------------------|------------------|
|
30 |
| [pair-ranker](https://huggingface.co/llm-blender/pair-ranker) (This model) | 128 | 128 | 384 |
|
31 |
| [pair-reward-model](https://huggingface.co/llm-blender/pair-reward-model/) | 1224 | 412 | 2048 |
|
32 |
|
33 |
|
34 |
+
### MixInstrut Performance
|
|
|
|
|
35 |
|
36 |
| **Methods** | BERTScore | BARTScore | BLEURT | GPT-Rank | Beat Vic(%) | Beat OA(%) | Top-1(%) | Top-2(%) | Top-3(%) |
|
37 |
|:-----------------:|:---------:|:---------:|:---------:|:--------:|:----------:|:----------:|:----------:|:----------:|:----------:|
|