Attention quantization: HQQ 4-bit, groupsize 64, compress zero, compress scale with groupsize 256
Experts quantization: HQQ 3-bit, groupsize 64, compress zero, compress scale with groupsize 128

Downloads last month: 6

Safetensors

Model size

6.91B params

Tensor type

FP16

I32

Inference Providers NEW

Text Generation

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support