iproskurina commited on
Commit
053daf8
·
verified ·
1 Parent(s): 617a019

Update README.md to include GPTQModel usage.

Browse files
Files changed (1) hide show
  1. README.md +0 -14
README.md CHANGED
@@ -85,17 +85,3 @@ model = GPTQModel.load(model_id)
85
  result = model.generate("Uncovering deep insights")[0] # tokens
86
  print(model.tokenizer.decode(result)) # string output
87
  ```
88
-
89
- ### Run the model with GPTQModel
90
-
91
- GPTQModel package: https://github.com/ModelCloud/GPTQModel
92
-
93
- ```
94
- pip install -v gptqmodel=="1.8.0" --no-build-isolation
95
- from gptqmodel import GPTQModel
96
-
97
- model_id = 'iproskurina/opt-125m-GPTQ-4bit-g128'
98
- model = GPTQModel.load(model_id)
99
- result = model.generate("Uncovering deep insights")[0] # tokens
100
- print(model.tokenizer.decode(result)) # string output
101
- ```
 
85
  result = model.generate("Uncovering deep insights")[0] # tokens
86
  print(model.tokenizer.decode(result)) # string output
87
  ```