Hugging Face's logo

medmekk
/
Qwen2.5-0.5B-Instruct.GGUF

GGUF
Inference Endpoints
imatrix
conversational
Model card Files Community
medmekk HF staff commited on
Commit
7c49df1
·
verified ·
1 Parent(s): 75e4a6c

Upload quantized models

Browse files
Files changed (22) hide show
  1. .gitattributes +20 -0
  2. Qwen2.5-0.5B-Instruct-IQ3_M_imat.gguf +3 -0
  3. Qwen2.5-0.5B-Instruct-IQ3_XXS_imat.gguf +3 -0
  4. Qwen2.5-0.5B-Instruct-IQ4_NL_imat.gguf +3 -0
  5. Qwen2.5-0.5B-Instruct-IQ4_XS_imat.gguf +3 -0
  6. Qwen2.5-0.5B-Instruct-Q2_K.gguf +3 -0
  7. Qwen2.5-0.5B-Instruct-Q3_K_L.gguf +3 -0
  8. Qwen2.5-0.5B-Instruct-Q3_K_M.gguf +3 -0
  9. Qwen2.5-0.5B-Instruct-Q3_K_S.gguf +3 -0
  10. Qwen2.5-0.5B-Instruct-Q4_0.gguf +3 -0
  11. Qwen2.5-0.5B-Instruct-Q4_K_M.gguf +3 -0
  12. Qwen2.5-0.5B-Instruct-Q4_K_M_imat.gguf +3 -0
  13. Qwen2.5-0.5B-Instruct-Q4_K_S.gguf +3 -0
  14. Qwen2.5-0.5B-Instruct-Q4_K_S_imat.gguf +3 -0
  15. Qwen2.5-0.5B-Instruct-Q5_0.gguf +3 -0
  16. Qwen2.5-0.5B-Instruct-Q5_K_M.gguf +3 -0
  17. Qwen2.5-0.5B-Instruct-Q5_K_M_imat.gguf +3 -0
  18. Qwen2.5-0.5B-Instruct-Q5_K_S.gguf +3 -0
  19. Qwen2.5-0.5B-Instruct-Q5_K_S_imat.gguf +3 -0
  20. Qwen2.5-0.5B-Instruct-Q6_K.gguf +3 -0
  21. Qwen2.5-0.5B-Instruct-Q8_0.gguf +3 -0
  22. README.md +34 -0
.gitattributes CHANGED
@@ -33,3 +33,23 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Qwen2.5-0.5B-Instruct-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
37
+ Qwen2.5-0.5B-Instruct-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Qwen2.5-0.5B-Instruct-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
39
+ Qwen2.5-0.5B-Instruct-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
40
+ Qwen2.5-0.5B-Instruct-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
41
+ Qwen2.5-0.5B-Instruct-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
42
+ Qwen2.5-0.5B-Instruct-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
43
+ Qwen2.5-0.5B-Instruct-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
44
+ Qwen2.5-0.5B-Instruct-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
45
+ Qwen2.5-0.5B-Instruct-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
46
+ Qwen2.5-0.5B-Instruct-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
47
+ Qwen2.5-0.5B-Instruct-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
48
+ Qwen2.5-0.5B-Instruct-IQ3_M_imat.gguf filter=lfs diff=lfs merge=lfs -text
49
+ Qwen2.5-0.5B-Instruct-IQ3_XXS_imat.gguf filter=lfs diff=lfs merge=lfs -text
50
+ Qwen2.5-0.5B-Instruct-Q4_K_M_imat.gguf filter=lfs diff=lfs merge=lfs -text
51
+ Qwen2.5-0.5B-Instruct-Q4_K_S_imat.gguf filter=lfs diff=lfs merge=lfs -text
52
+ Qwen2.5-0.5B-Instruct-IQ4_NL_imat.gguf filter=lfs diff=lfs merge=lfs -text
53
+ Qwen2.5-0.5B-Instruct-IQ4_XS_imat.gguf filter=lfs diff=lfs merge=lfs -text
54
+ Qwen2.5-0.5B-Instruct-Q5_K_M_imat.gguf filter=lfs diff=lfs merge=lfs -text
55
+ Qwen2.5-0.5B-Instruct-Q5_K_S_imat.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-0.5B-Instruct-IQ3_M_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2ca80f6dde976fc3434a638e5764d9acefcb33419b82ea3206742560f2ef7a18
3
+ size 342752544
Qwen2.5-0.5B-Instruct-IQ3_XXS_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:677df623100dafaf967baf91c299cffd1a3102ea02745eafd5cdccdf9a8ce571
3
+ size 333704736
Qwen2.5-0.5B-Instruct-IQ4_NL_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c76060404253b50611ee0c2ab732591608446942fdba49cf6d4da6c681b65aa2
3
+ size 352671264
Qwen2.5-0.5B-Instruct-IQ4_XS_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fd91514a39ca40a616eb36479514c6762d7c647ec9b182a8e51e4185ca77d01d
3
+ size 349402656
Qwen2.5-0.5B-Instruct-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0183050b0aa6a58c451fb558d3fdfa550c3dd6ba835561805778d30bdd79e44a
3
+ size 338607424
Qwen2.5-0.5B-Instruct-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:acefd39598b7d1e91c7ac2ee9ae0fa1fc3efde0045d6c6f5371163b4653ed145
3
+ size 369358144
Qwen2.5-0.5B-Instruct-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0865856b69d39ef0aa29024e6ceef97799aeba3fc72d8bdc5cd363017fc2b444
3
+ size 355466560
Qwen2.5-0.5B-Instruct-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4b3c2f18a215e8b0dcdbd975d672bd9c2c052281c7767feb0f6cc1b389757788
3
+ size 338263360
Qwen2.5-0.5B-Instruct-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3a8c81c78cc8ce6e7bf2bd34b87dd86ab9e929a1bb7ce907c41ee5014fdb4451
3
+ size 352154944
Qwen2.5-0.5B-Instruct-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:750f8f144f0504208add7897f01c7d2350a7363d8855eab59e137a1041e90394
3
+ size 397807936
Qwen2.5-0.5B-Instruct-Q4_K_M_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7731735104fbed4fca2ea4ff2f8f349d7722e982714f6706dcb760b0b6685287
3
+ size 397808160
Qwen2.5-0.5B-Instruct-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:62c51f95d2a1dc8a196faf41bc0a1952b1a49e92e9476afe8b131086954faf1b
3
+ size 385471808
Qwen2.5-0.5B-Instruct-Q4_K_S_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8ca2d29880b700dd846fbb747805268584c834bc9de43fad71c18e0c3ee44bcd
3
+ size 385472032
Qwen2.5-0.5B-Instruct-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f4367efbfe662893372a747982d7fdfcf03907780efbc56ce85405fdd69a7236
3
+ size 396883264
Qwen2.5-0.5B-Instruct-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3450d966e73cd5ae9c92f493ed56fb60b5d6dbed0f945652e6cc6ce54ea143d3
3
+ size 420086080
Qwen2.5-0.5B-Instruct-Q5_K_M_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2492899614384d41f400cf45d84b5ca3800d7d24ec481151597f09d1928e4653
3
+ size 420086304
Qwen2.5-0.5B-Instruct-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2036737cc5df9574995375559c760d4e731885fe7fa8f4d12670a6aadd33e929
3
+ size 412710208
Qwen2.5-0.5B-Instruct-Q5_K_S_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1b911ee5126180929b9d1ca792dc3db5ba972dbe207f78c8999ec68df7d0beac
3
+ size 412710432
Qwen2.5-0.5B-Instruct-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a9c7b83b92ade06829df7abc0d85161240f51b008da377482ba411307602ca0e
3
+ size 505736512
Qwen2.5-0.5B-Instruct-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:673eb9fba744c9686488be1ddbb0fbe07b9cf18373bc87c4b1fcab063a7d2aae
3
+ size 531068224
README.md ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ # medmekk/Qwen2.5-0.5B-Instruct.GGUF
3
+ GGUF quantized versions of [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct)
4
+
5
+ ## Available Formats:
6
+ - `Q2_K`: Qwen2.5-0.5B-Instruct-Q2_K.gguf
7
+ - `Q3_K_S`: Qwen2.5-0.5B-Instruct-Q3_K_S.gguf
8
+ - `Q3_K_M`: Qwen2.5-0.5B-Instruct-Q3_K_M.gguf
9
+ - `Q3_K_L`: Qwen2.5-0.5B-Instruct-Q3_K_L.gguf
10
+ - `Q4_0`: Qwen2.5-0.5B-Instruct-Q4_0.gguf
11
+ - `Q4_K_S`: Qwen2.5-0.5B-Instruct-Q4_K_S.gguf
12
+ - `Q4_K_M`: Qwen2.5-0.5B-Instruct-Q4_K_M.gguf
13
+ - `Q5_0`: Qwen2.5-0.5B-Instruct-Q5_0.gguf
14
+ - `Q5_K_S`: Qwen2.5-0.5B-Instruct-Q5_K_S.gguf
15
+ - `Q5_K_M`: Qwen2.5-0.5B-Instruct-Q5_K_M.gguf
16
+ - `Q6_K`: Qwen2.5-0.5B-Instruct-Q6_K.gguf
17
+ - `Q8_0`: Qwen2.5-0.5B-Instruct-Q8_0.gguf
18
+ - `IQ3_M_IMAT`: Qwen2.5-0.5B-Instruct-IQ3_M_imat.gguf
19
+ - `IQ3_XXS_IMAT`: Qwen2.5-0.5B-Instruct-IQ3_XXS_imat.gguf
20
+ - `Q4_K_M_IMAT`: Qwen2.5-0.5B-Instruct-Q4_K_M_imat.gguf
21
+ - `Q4_K_S_IMAT`: Qwen2.5-0.5B-Instruct-Q4_K_S_imat.gguf
22
+ - `IQ4_NL_IMAT`: Qwen2.5-0.5B-Instruct-IQ4_NL_imat.gguf
23
+ - `IQ4_XS_IMAT`: Qwen2.5-0.5B-Instruct-IQ4_XS_imat.gguf
24
+ - `Q5_K_M_IMAT`: Qwen2.5-0.5B-Instruct-Q5_K_M_imat.gguf
25
+ - `Q5_K_S_IMAT`: Qwen2.5-0.5B-Instruct-Q5_K_S_imat.gguf
26
+
27
+ ## Usage with llama.cpp:
28
+ ```bash
29
+ # CLI:
30
+ llama-cli --hf-repo medmekk/Qwen2.5-0.5B-Instruct.GGUF --hf-file MODEL_FILE -p "Your prompt"
31
+
32
+ # Server:
33
+ llama-server --hf-repo medmekk/Qwen2.5-0.5B-Instruct.GGUF --hf-file MODEL_FILE -c 2048
34
+ ```