Qwen/QwQ-32B-Preview GGUF Quantizations ๐Ÿš€

Featherless AI Quants

Optimized GGUF quantization files for enhanced model performance

Powered by Featherless AI - run any model you'd like for a simple small fee.


Available Quantizations ๐Ÿ“Š

Quantization Type File Size
IQ4_XS Qwen-QwQ-32B-Preview-IQ4_XS.gguf 17042.26 MB
Q2_K Qwen-QwQ-32B-Preview-Q2_K.gguf 11742.69 MB
Q3_K_L Qwen-QwQ-32B-Preview-Q3_K_L.gguf 16448.10 MB
Q3_K_M Qwen-QwQ-32B-Preview-Q3_K_M.gguf 15196.85 MB
Q3_K_S Qwen-QwQ-32B-Preview-Q3_K_S.gguf 13725.60 MB
Q4_K_M Qwen-QwQ-32B-Preview-Q4_K_M.gguf 18931.71 MB
Q4_K_S Qwen-QwQ-32B-Preview-Q4_K_S.gguf 17914.21 MB
Q5_K_M Qwen-QwQ-32B-Preview-Q5_K_M.gguf 22184.52 MB
Q5_K_S Qwen-QwQ-32B-Preview-Q5_K_S.gguf 21589.52 MB
Q6_K Qwen-QwQ-32B-Preview-Q6_K 25640.64 MB (folder)
Q8_0 Qwen-QwQ-32B-Preview-Q8_0 33207.78 MB (folder)

โšก Powered by Featherless AI

Key Features

  • ๐Ÿ”ฅ Instant Hosting - Deploy any Llama model on HuggingFace instantly
  • ๐Ÿ› ๏ธ Zero Infrastructure - No server setup or maintenance required
  • ๐Ÿ“š Vast Compatibility - Support for 2400+ models and counting
  • ๐Ÿ’Ž Affordable Pricing - Starting at just $10/month

Links:
Get Started | Documentation | Models

Downloads last month
1,969
GGUF
Model size
32.8B params
Architecture
qwen2

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The model has no library tag.

Model tree for featherless-ai-quants/Qwen-QwQ-32B-Preview-GGUF

Base model

Qwen/Qwen2.5-32B
Quantized
(116)
this model