Hugging Face's logo

nitsuai
/
Bio-Medical-MultiModal-Llama-3-8B-V1-i1-GGUF

Transformers
GGUF
English
Generated from Trainer
Healthcare & Lifesciences
BioMed
Medical
Multimodal
Vision
Text
Contact Doctor
MiniCPM
Llama 3
imatrix
conversational
Model card Files Community
nitsuai mradermacher commited on
Commit
89bb534
·
verified ·
0 Parent(s):

Duplicate from mradermacher/Bio-Medical-MultiModal-Llama-3-8B-V1-i1-GGUF

Browse files

Co-authored-by: team mradermacher <[email protected]>

Files changed (25) hide show
  1. .gitattributes +58 -0
  2. Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ1_M.gguf +3 -0
  3. Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ1_S.gguf +3 -0
  4. Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ2_M.gguf +3 -0
  5. Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ2_S.gguf +3 -0
  6. Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ2_XS.gguf +3 -0
  7. Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ2_XXS.gguf +3 -0
  8. Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ3_M.gguf +3 -0
  9. Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ3_S.gguf +3 -0
  10. Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ3_XS.gguf +3 -0
  11. Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ3_XXS.gguf +3 -0
  12. Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ4_XS.gguf +3 -0
  13. Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q2_K.gguf +3 -0
  14. Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q2_K_S.gguf +3 -0
  15. Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q3_K_L.gguf +3 -0
  16. Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q3_K_M.gguf +3 -0
  17. Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q3_K_S.gguf +3 -0
  18. Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q4_0.gguf +3 -0
  19. Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q4_K_M.gguf +3 -0
  20. Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q4_K_S.gguf +3 -0
  21. Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q5_K_M.gguf +3 -0
  22. Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q5_K_S.gguf +3 -0
  23. Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q6_K.gguf +3 -0
  24. README.md +87 -0
  25. imatrix.dat +3 -0
.gitattributes ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ imatrix.dat filter=lfs diff=lfs merge=lfs -text
37
+ Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
39
+ Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
40
+ Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
41
+ Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
42
+ Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
43
+ Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
44
+ Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
45
+ Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
46
+ Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
47
+ Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
48
+ Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
49
+ Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
50
+ Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
51
+ Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
52
+ Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
53
+ Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
54
+ Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
55
+ Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
56
+ Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
57
+ Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
58
+ Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ1_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e985cd2ab082c9eaa2d674df93554f402fdfd6f298cd9b78a2c2703619b36fb8
3
+ size 2161972160
Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ1_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2d6964be423ee6b74437fd799438e10ac6b68afe8f53fac389b42b6de23a497e
3
+ size 2019627968
Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ2_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2da819090f13782b3ff799f1b4036e4cc784c1f8b857eb6fa7696a62a9e31b7a
3
+ size 2948281280
Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ2_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7f6ae52a5a95728293c39a722f4994569c4a3b26f201a05a15bbb154e91c56ed
3
+ size 2758489024
Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ2_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6db3387b6eeb8bad083ecec850e124299c778de553880d4dc1fedb6a8f6c55b4
3
+ size 2605781952
Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ2_XXS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bfdcd505575c093ecaef3cca63f2b20e89146934f00c9cd84d1a65e301d536a5
3
+ size 2399212480
Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ3_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fcc0881dd0597469a96b9308801c9d10aa9092d7d242ea462d87c6dbce11a71a
3
+ size 3784823744
Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ3_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bfa87aba82dea278040587015950be9709d60c67dbf130eea2174cd2130839bf
3
+ size 3682325440
Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ3_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:63344581482c06604ec7c9f323c30da5c65d010aab84403e4522d6cf8e27e16b
3
+ size 3518747584
Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ3_XXS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e06b614ca27d0786bf48bfbf566fd4b2690acc6ee40d9259ec48031bb6ac4d45
3
+ size 3274912704
Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ4_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6591ac29a01f9571857a3ca68ec3f4d00263d16a06a858c4e0c8cbe5425dc82b
3
+ size 4447663040
Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b03b36965e78941e55b4cf515acbf37b01506bc3ba7a703baa275ac4af82d2b2
3
+ size 3179131840
Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q2_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:59624bb2d441e307e29587042f0b41938fc0e9d6d9e7b97ef50f7d18a38c0b7f
3
+ size 2988815296
Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:86a795bc94848768457fd5d9f1c414d74160103700a620d654d169ccf7970d5c
3
+ size 4321956800
Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4dfe164c55d0e54384183e624bea94e2155699e08ba6beb88a43cadec6adee0b
3
+ size 4018918336
Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b4c96d500510c16690a5896beeff7eb1eea6952715e003cf2787cfa157d7f810
3
+ size 3664499648
Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a307ccbb17361f8bceba5f58f6c874b958feafed65abce5875b179da96f21a7d
3
+ size 4675892160
Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3db4c136f0e5f0f2b708e4b226809a2ef0cb0dcee6423ef37a7862f2765cd02f
3
+ size 4920734656
Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:67479322408e04181f5dec40fa15c826422fe47d5f7a09953b343fecd38d5914
3
+ size 4692669376
Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fe542eeb520940007c6b41aa762db8f4695be31ef29d6d32ceea863f84a4da09
3
+ size 5732987840
Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:90d341bdf30af30147890b7b6e3f5c22eeb62fcf126f128ea5120360bac3a541
3
+ size 5599294400
Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bc6aebd42c578cfc7d3c4fb6aabb7e1937f57d3fba2f12f9569201c07c6b10a7
3
+ size 6596006848
README.md ADDED
@@ -0,0 +1,87 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: ContactDoctor/Bio-Medical-MultiModal-Llama-3-8B-V1
3
+ datasets:
4
+ - collaiborateorg/BioMedData
5
+ language:
6
+ - en
7
+ library_name: transformers
8
+ license: other
9
+ quantized_by: mradermacher
10
+ tags:
11
+ - generated_from_trainer
12
+ - Healthcare & Lifesciences
13
+ - BioMed
14
+ - Medical
15
+ - Multimodal
16
+ - Vision
17
+ - Text
18
+ - Contact Doctor
19
+ - MiniCPM
20
+ - Llama 3
21
+ ---
22
+ ## About
23
+
24
+ <!-- ### quantize_version: 2 -->
25
+ <!-- ### output_tensor_quantised: 1 -->
26
+ <!-- ### convert_type: hf -->
27
+ <!-- ### vocab_type: -->
28
+ <!-- ### tags: nicoboss -->
29
+ weighted/imatrix quants of https://huggingface.co/ContactDoctor/Bio-Medical-MultiModal-Llama-3-8B-V1
30
+
31
+ <!-- provided-files -->
32
+ static quants are available at https://huggingface.co/mradermacher/Bio-Medical-MultiModal-Llama-3-8B-V1-GGUF
33
+ ## Usage
34
+
35
+ If you are unsure how to use GGUF files, refer to one of [TheBloke's
36
+ READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
37
+ more details, including on how to concatenate multi-part files.
38
+
39
+ ## Provided Quants
40
+
41
+ (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
42
+
43
+ | Link | Type | Size/GB | Notes |
44
+ |:-----|:-----|--------:|:------|
45
+ | [GGUF](https://huggingface.co/mradermacher/Bio-Medical-MultiModal-Llama-3-8B-V1-i1-GGUF/resolve/main/Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ1_S.gguf) | i1-IQ1_S | 2.1 | for the desperate |
46
+ | [GGUF](https://huggingface.co/mradermacher/Bio-Medical-MultiModal-Llama-3-8B-V1-i1-GGUF/resolve/main/Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ1_M.gguf) | i1-IQ1_M | 2.3 | mostly desperate |
47
+ | [GGUF](https://huggingface.co/mradermacher/Bio-Medical-MultiModal-Llama-3-8B-V1-i1-GGUF/resolve/main/Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.5 | |
48
+ | [GGUF](https://huggingface.co/mradermacher/Bio-Medical-MultiModal-Llama-3-8B-V1-i1-GGUF/resolve/main/Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.7 | |
49
+ | [GGUF](https://huggingface.co/mradermacher/Bio-Medical-MultiModal-Llama-3-8B-V1-i1-GGUF/resolve/main/Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ2_S.gguf) | i1-IQ2_S | 2.9 | |
50
+ | [GGUF](https://huggingface.co/mradermacher/Bio-Medical-MultiModal-Llama-3-8B-V1-i1-GGUF/resolve/main/Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ2_M.gguf) | i1-IQ2_M | 3.0 | |
51
+ | [GGUF](https://huggingface.co/mradermacher/Bio-Medical-MultiModal-Llama-3-8B-V1-i1-GGUF/resolve/main/Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q2_K_S.gguf) | i1-Q2_K_S | 3.1 | very low quality |
52
+ | [GGUF](https://huggingface.co/mradermacher/Bio-Medical-MultiModal-Llama-3-8B-V1-i1-GGUF/resolve/main/Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q2_K.gguf) | i1-Q2_K | 3.3 | IQ3_XXS probably better |
53
+ | [GGUF](https://huggingface.co/mradermacher/Bio-Medical-MultiModal-Llama-3-8B-V1-i1-GGUF/resolve/main/Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 3.4 | lower quality |
54
+ | [GGUF](https://huggingface.co/mradermacher/Bio-Medical-MultiModal-Llama-3-8B-V1-i1-GGUF/resolve/main/Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ3_XS.gguf) | i1-IQ3_XS | 3.6 | |
55
+ | [GGUF](https://huggingface.co/mradermacher/Bio-Medical-MultiModal-Llama-3-8B-V1-i1-GGUF/resolve/main/Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q3_K_S.gguf) | i1-Q3_K_S | 3.8 | IQ3_XS probably better |
56
+ | [GGUF](https://huggingface.co/mradermacher/Bio-Medical-MultiModal-Llama-3-8B-V1-i1-GGUF/resolve/main/Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ3_S.gguf) | i1-IQ3_S | 3.8 | beats Q3_K* |
57
+ | [GGUF](https://huggingface.co/mradermacher/Bio-Medical-MultiModal-Llama-3-8B-V1-i1-GGUF/resolve/main/Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ3_M.gguf) | i1-IQ3_M | 3.9 | |
58
+ | [GGUF](https://huggingface.co/mradermacher/Bio-Medical-MultiModal-Llama-3-8B-V1-i1-GGUF/resolve/main/Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q3_K_M.gguf) | i1-Q3_K_M | 4.1 | IQ3_S probably better |
59
+ | [GGUF](https://huggingface.co/mradermacher/Bio-Medical-MultiModal-Llama-3-8B-V1-i1-GGUF/resolve/main/Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q3_K_L.gguf) | i1-Q3_K_L | 4.4 | IQ3_M probably better |
60
+ | [GGUF](https://huggingface.co/mradermacher/Bio-Medical-MultiModal-Llama-3-8B-V1-i1-GGUF/resolve/main/Bio-Medical-MultiModal-Llama-3-8B-V1.i1-IQ4_XS.gguf) | i1-IQ4_XS | 4.5 | |
61
+ | [GGUF](https://huggingface.co/mradermacher/Bio-Medical-MultiModal-Llama-3-8B-V1-i1-GGUF/resolve/main/Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q4_0.gguf) | i1-Q4_0 | 4.8 | fast, low quality |
62
+ | [GGUF](https://huggingface.co/mradermacher/Bio-Medical-MultiModal-Llama-3-8B-V1-i1-GGUF/resolve/main/Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q4_K_S.gguf) | i1-Q4_K_S | 4.8 | optimal size/speed/quality |
63
+ | [GGUF](https://huggingface.co/mradermacher/Bio-Medical-MultiModal-Llama-3-8B-V1-i1-GGUF/resolve/main/Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q4_K_M.gguf) | i1-Q4_K_M | 5.0 | fast, recommended |
64
+ | [GGUF](https://huggingface.co/mradermacher/Bio-Medical-MultiModal-Llama-3-8B-V1-i1-GGUF/resolve/main/Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q5_K_S.gguf) | i1-Q5_K_S | 5.7 | |
65
+ | [GGUF](https://huggingface.co/mradermacher/Bio-Medical-MultiModal-Llama-3-8B-V1-i1-GGUF/resolve/main/Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q5_K_M.gguf) | i1-Q5_K_M | 5.8 | |
66
+ | [GGUF](https://huggingface.co/mradermacher/Bio-Medical-MultiModal-Llama-3-8B-V1-i1-GGUF/resolve/main/Bio-Medical-MultiModal-Llama-3-8B-V1.i1-Q6_K.gguf) | i1-Q6_K | 6.7 | practically like static Q6_K |
67
+
68
+ Here is a handy graph by ikawrakow comparing some lower-quality quant
69
+ types (lower is better):
70
+
71
+ ![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
72
+
73
+ And here are Artefact2's thoughts on the matter:
74
+ https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
75
+
76
+ ## FAQ / Model Request
77
+
78
+ See https://huggingface.co/mradermacher/model_requests for some answers to
79
+ questions you might have and/or if you want some other model quantized.
80
+
81
+ ## Thanks
82
+
83
+ I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
84
+ me use its servers and providing upgrades to my workstation to enable
85
+ this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
86
+
87
+ <!-- end -->
imatrix.dat ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ef56569eca532eec9ed72f4bebc929643391c8bf469d44b94c01d67322f4667f
3
+ size 4988157