Hibrid-Llama-Linear GGUF Quantized Models

Technical Details

  • Quantization Tool: llama.cpp
  • Version: version: 5126 (307bfa25)

Model Information

Available Files

🚀 Download 🔢 Type 📝 Description
Download Q3 K M Small, acceptable quality
Download Q4 0 Standard 4-bit (fast on ARM)
Download Q4 K M 4-bit balanced (recommended default)
Download Q5 K M 5-bit best (recommended HQ option)
Download Q6 K 6-bit near-perfect (premium quality)
Download Q8 0 8-bit maximum (overkill for most)

💡 Q4 K M provides the best balance for most use cases

Downloads last month
59
GGUF
Model size
8.03B params
Architecture
llama
Hardware compatibility
Log In to view the estimation

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for matrixportal/Hibrid-Llama-Linear-GGUF

Collection including matrixportal/Hibrid-Llama-Linear-GGUF