Commit
·
613ae91
1
Parent(s):
122d2d5
Update README.md
Browse files
README.md
CHANGED
@@ -3,6 +3,11 @@ library_name: peft
|
|
3 |
---
|
4 |
## Examples
|
5 |
As you can see from examples bellow the output is far from ideal, and far from simple GPT/LLama2 prompt without finetuning.
|
|
|
|
|
|
|
|
|
|
|
6 |
The low quality is probably caused by very low volume of training data - 100 rows, low amount of combinations which can be then hardly generalized.
|
7 |
|
8 |
### Example 1
|
@@ -39,8 +44,10 @@ Repairs:
|
|
39 |
Repairs:
|
40 |
- The verb "gemachen" is in the wrong form, it should be "gemacht" - this is the third person singular of the verb "machen" in the past tense.#### End AI %}
|
41 |
Repairs:
|
|
|
42 |
## Training procedure
|
43 |
-
Trained on: 1x RTX A6000
|
|
|
44 |
8 Epochs, cca 25 minutes, Loss: 0.36
|
45 |
|
46 |
The following `bitsandbytes` quantization config was used during training:
|
|
|
3 |
---
|
4 |
## Examples
|
5 |
As you can see from examples bellow the output is far from ideal, and far from simple GPT/LLama2 prompt without finetuning.
|
6 |
+
Quality issues:
|
7 |
+
- the corrected sentences are containing some minor errors
|
8 |
+
- the explanations of corrections are far from optimal, they often refer to some additional words as mented in explanations which are not necessary
|
9 |
+
- the output repeats itself
|
10 |
+
- the output list is sometimes broken into two lists
|
11 |
The low quality is probably caused by very low volume of training data - 100 rows, low amount of combinations which can be then hardly generalized.
|
12 |
|
13 |
### Example 1
|
|
|
44 |
Repairs:
|
45 |
- The verb "gemachen" is in the wrong form, it should be "gemacht" - this is the third person singular of the verb "machen" in the past tense.#### End AI %}
|
46 |
Repairs:
|
47 |
+
|
48 |
## Training procedure
|
49 |
+
Trained on: 1x RTX A6000
|
50 |
+
Used: 30GB Ram, 47GB GPU, 130GB Disc
|
51 |
8 Epochs, cca 25 minutes, Loss: 0.36
|
52 |
|
53 |
The following `bitsandbytes` quantization config was used during training:
|