Hugging Face's logo

thrunlab
/
sparse_llama_7b_hf2_refined_web_50p_2024-03-27

Text Generation
Transformers
Safetensors
sparse_llama
Generated from Trainer
custom_code
Model card Files Community
lukeleeai commited on
Commit
7b766c7
·
verified ·
1 Parent(s): 54fcd04

End of training

Browse files
Files changed (4) hide show
  1. README.md +16 -6
  2. model-00001-of-00003.safetensors +1 -1
  3. model-00002-of-00003.safetensors +1 -1
  4. model-00003-of-00003.safetensors +1 -1
README.md CHANGED
@@ -14,7 +14,7 @@ should probably proofread and complete it, then remove this comment. -->
14
 
15
  This model is a fine-tuned version of [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) on the None dataset.
16
  It achieves the following results on the evaluation set:
17
- - Loss: 2.0930
18
 
19
  ## Model description
20
 
@@ -44,7 +44,7 @@ The following hyperparameters were used during training:
44
  - total_eval_batch_size: 16
45
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
46
  - lr_scheduler_type: linear
47
- - training_steps: 600
48
 
49
  ### Training results
50
 
@@ -70,10 +70,20 @@ The following hyperparameters were used during training:
70
  | 2.1799 | 0.14 | 450 | 2.2267 |
71
  | 2.252 | 0.15 | 475 | 2.2250 |
72
  | 2.2588 | 0.16 | 500 | 2.2262 |
73
- | 2.167 | 0.17 | 525 | 2.2245 |
74
- | 2.162 | 0.18 | 550 | 2.2262 |
75
- | 2.2792 | 0.18 | 575 | 2.2263 |
76
- | 2.1621 | 0.19 | 600 | 2.2264 |
 
 
 
 
 
 
 
 
 
 
77
 
78
 
79
  ### Framework versions
 
14
 
15
  This model is a fine-tuned version of [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) on the None dataset.
16
  It achieves the following results on the evaluation set:
17
+ - Loss: 2.0846
18
 
19
  ## Model description
20
 
 
44
  - total_eval_batch_size: 16
45
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
46
  - lr_scheduler_type: linear
47
+ - training_steps: 850
48
 
49
  ### Training results
50
 
 
70
  | 2.1799 | 0.14 | 450 | 2.2267 |
71
  | 2.252 | 0.15 | 475 | 2.2250 |
72
  | 2.2588 | 0.16 | 500 | 2.2262 |
73
+ | 2.1677 | 0.17 | 525 | 2.2271 |
74
+ | 2.163 | 0.18 | 550 | 2.2264 |
75
+ | 2.2783 | 0.18 | 575 | 2.2251 |
76
+ | 2.1625 | 0.19 | 600 | 2.2253 |
77
+ | 2.1906 | 0.2 | 625 | 2.2251 |
78
+ | 2.2748 | 0.21 | 650 | 2.2251 |
79
+ | 2.171 | 0.22 | 675 | 2.2249 |
80
+ | 2.1929 | 0.22 | 700 | 2.2252 |
81
+ | 2.2203 | 0.23 | 725 | 2.2232 |
82
+ | 2.1143 | 0.24 | 750 | 2.2239 |
83
+ | 2.1958 | 0.25 | 775 | 2.2242 |
84
+ | 2.2491 | 0.26 | 800 | 2.2227 |
85
+ | 2.1991 | 0.26 | 825 | 2.2239 |
86
+ | 2.1555 | 0.27 | 850 | 2.2228 |
87
 
88
 
89
  ### Framework versions
model-00001-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:3ed32d5a3c7439c5404b6b924ab3ef589810606bacef6bd3d5283a2e868b8770
3
  size 4938985352
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d0f0c29fadca907bff03ad1cb5a8b56f1557c8bf284942a5c6a72fa9ae3c3243
3
  size 4938985352
model-00002-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c5422a54a5f08b57d9c24aaa9e805009b856f3ba6d7cae6496a8b81ab21794a5
3
  size 4947390880
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aedc882348dfa818e03fc17e96737dedb198bdd1f72dd721955d3974ed4edf12
3
  size 4947390880
model-00003-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:24a5dc3f72ab8bb0fbb988d7f75571e325b8bd13ab4742a02aa3d2a6e3681dbd
3
  size 3590488816
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d9c712c04379a49c3346152a36a60cce244019bd33a929a0282e065f7c551d41
3
  size 3590488816