AnirudhRajagopalan1201 commited on
Commit
bffd58c
·
verified ·
1 Parent(s): 3f63300

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -4
README.md CHANGED
@@ -3,10 +3,11 @@ library_name: transformers
3
  datasets:
4
  - roneneldan/TinyStories
5
  ---
6
- Model trained on the TinyStories Dataset, replicating https://arxiv.org/abs/2305.07759
7
- Based on GPT-Neo architecture.
8
 
9
- hyperparams used to train this model:
 
10
  ```
11
  "batch_size": 32,
12
  "block_size": 256,
@@ -23,7 +24,8 @@ hyperparams used to train this model:
23
  "warmup_tokens": 10000,
24
  "gradient_accumulation_steps": 8
25
  ```
26
- ------ EXAMPLE USAGE ---
 
27
  ```py
28
  !pip install --quiet transformers
29
  from transformers import AutoModelForCausalLM, AutoTokenizer
 
3
  datasets:
4
  - roneneldan/TinyStories
5
  ---
6
+ ---
7
+ Model trained on the TinyStories Dataset, replicating https://arxiv.org/abs/2305.07759, based on GPT-Neo architecture.
8
 
9
+ ---
10
+ Hyperparams used to train this model:
11
  ```
12
  "batch_size": 32,
13
  "block_size": 256,
 
24
  "warmup_tokens": 10000,
25
  "gradient_accumulation_steps": 8
26
  ```
27
+ ---
28
+ EXAMPLE USAGE
29
  ```py
30
  !pip install --quiet transformers
31
  from transformers import AutoModelForCausalLM, AutoTokenizer