Update README.md
Browse files
README.md
CHANGED
@@ -3,10 +3,11 @@ library_name: transformers
|
|
3 |
datasets:
|
4 |
- roneneldan/TinyStories
|
5 |
---
|
6 |
-
|
7 |
-
|
8 |
|
9 |
-
|
|
|
10 |
```
|
11 |
"batch_size": 32,
|
12 |
"block_size": 256,
|
@@ -23,7 +24,8 @@ hyperparams used to train this model:
|
|
23 |
"warmup_tokens": 10000,
|
24 |
"gradient_accumulation_steps": 8
|
25 |
```
|
26 |
-
|
|
|
27 |
```py
|
28 |
!pip install --quiet transformers
|
29 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
3 |
datasets:
|
4 |
- roneneldan/TinyStories
|
5 |
---
|
6 |
+
---
|
7 |
+
Model trained on the TinyStories Dataset, replicating https://arxiv.org/abs/2305.07759, based on GPT-Neo architecture.
|
8 |
|
9 |
+
---
|
10 |
+
Hyperparams used to train this model:
|
11 |
```
|
12 |
"batch_size": 32,
|
13 |
"block_size": 256,
|
|
|
24 |
"warmup_tokens": 10000,
|
25 |
"gradient_accumulation_steps": 8
|
26 |
```
|
27 |
+
---
|
28 |
+
EXAMPLE USAGE
|
29 |
```py
|
30 |
!pip install --quiet transformers
|
31 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|