Update README.md

README.md

@@ -11,7 +11,7 @@ base_model:
 pipeline_tag: text-generation
 ---
 
-# rwkv7-168M-pile
+# rwkv7-421M-pile
 
 <!-- Provide a quick summary of what the model is/does. -->
 
@@ -29,7 +29,7 @@ This is RWKV-7 model under flash-linear attention format.
 - **Model type:** RWKV7
 - **Language(s) (NLP):** English
 - **License:** Apache-2.0
-- **Parameter count:** 168M
+- **Parameter count:** 421M
 - **Tokenizer:** GPT-NeoX 20B tokenizer
 
 ### Model Sources
@@ -38,7 +38,6 @@ This is RWKV-7 model under flash-linear attention format.
 
 - **Repository:** https://github.com/fla-org/flash-linear-attention ; https://github.com/BlinkDL/RWKV-LM
 - **Paper:** With in Progress
-- **Weights:** Converted from https://modelscope.cn/models/RWKV/rwkv-7-pile/file/view/master?fileName=RWKV-x070-Pile-168M-20241120-ctx4096.pth
 
 ## Uses
 
@@ -56,8 +55,8 @@ pip install 'transformers>=4.48.0'
 You can use this model just as any other HuggingFace models:
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
-model = AutoModelForCausalLM.from_pretrained('fla-hub/rwkv7-168M-pile', trust_remote_code=True)
-tokenizer = AutoTokenizer.from_pretrained('fla-hub/rwkv7-168M-pile', trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained('fla-hub/rwkv7-421M-pile', trust_remote_code=True)
+tokenizer = AutoTokenizer.from_pretrained('fla-hub/rwkv7-421M-pile', trust_remote_code=True)
 ```
 
 ## Training Details
@@ -74,9 +73,9 @@ This model is trained on the Pile with a total of 332 billion tokens.
 
 #### Metrics
 
-`lambada_openai`: ppl 14.2 acc 45.6%
+`lambada_openai`: ppl 7.21 acc 57.9%
 
-`piqa`: acc 65.5%
+`piqa`: acc 69.2%
 
 ## FAQ
 Q: safetensors metadata is none.