Hugging Face's logo

rombodawg
/
Rombos-LLM-V2.5-Qwen-72b

Text Generation
Transformers
Safetensors
qwen2
conversational
Eval Results
text-generation-inference
Model card Files Community
6

Can you make a 2.25bpw quantization for this model?

#4
by xldistance - opened
Discussion
xldistance
•
edited Dec 10, 2024

@LoneStriker 24GB of video memory can only run 2.4bpw quantization

Sign up or log in to comment