Update README.md
Browse files
README.md
CHANGED
@@ -38,14 +38,14 @@ Our DeepSeek-V3-0324 GGUFs allow you to run the model in llama.cpp, LMStudio, Op
|
|
38 |
|
39 |
Includes 1-4-bit Dynamic versions, which yields better accuracy and results than standard quantization.
|
40 |
|
41 |
-
| MoE Bits
|
42 |
-
|
43 |
-
| 1.78bit (prelim)
|
44 |
-
| 1.93bit (prelim)
|
45 |
-
| 2.42bit
|
46 |
-
| 2.71bit
|
47 |
-
| 3.5bit
|
48 |
-
| 4.5bit
|
49 |
|
50 |
Prelim = preliminary - through our testing, they're generally fine but sometimes don't produce the best code and so more work/testing needs to be done.
|
51 |
2.71bit was found to be the best in terms of performance/size and produces code that is great and works well. 2.42bit was also found to pass all our tests.
|
|
|
38 |
|
39 |
Includes 1-4-bit Dynamic versions, which yields better accuracy and results than standard quantization.
|
40 |
|
41 |
+
| **MoE Bits** | **Type** | **Disk Size** | **Link** | **Details** |
|
42 |
+
|--------------------|------------|----------------|----------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------|
|
43 |
+
| 1.78bit (prelim) | IQ1_S | **192GB** | [Link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF-UD/tree/main/UD-IQ1_S) | down_proj in MoE mixture of 2.06/1.78bit |
|
44 |
+
| 1.93bit (prelim) | IQ1_M | **200GB** | [Link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF-UD/tree/main/UD-IQ1_M) | down_proj in MoE mixture of 2.06/1.93bit |
|
45 |
+
| 2.42bit | IQ2_XXS | **215GB** | [Link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF-UD/tree/main/UD-IQ2_XXS) | down_proj in MoE all 2.42bit |
|
46 |
+
| 2.71bit | Q2_K_XL | **250GB** | [Link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF-UD/tree/main/UD-Q2_K_XL) | down_proj in MoE mixture of 3.5/2.71bit |
|
47 |
+
| 3.5bit | Q3_K_XL | **296GB** | [Link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF-UD/tree/main/UD-Q3_K_XL) | down_proj in MoE mixture of 4.5/3.5bit |
|
48 |
+
| 4.5bit | Q4_K_XL | **384GB** | [Link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF-UD/tree/main/UD-Q4_K_XL) | down_proj in MoE mixture of 5.5/4.5bit |
|
49 |
|
50 |
Prelim = preliminary - through our testing, they're generally fine but sometimes don't produce the best code and so more work/testing needs to be done.
|
51 |
2.71bit was found to be the best in terms of performance/size and produces code that is great and works well. 2.42bit was also found to pass all our tests.
|