Update README.md
Browse files
README.md
CHANGED
@@ -6,23 +6,34 @@ library_name: transformers
|
|
6 |
|
7 |

|
8 |
|
9 |
-
**New Version
|
10 |
|
11 |
Credit to Fernando Fernandes and Eric Hartford for their project [laserRMT](https://github.com/cognitivecomputations/laserRMT)
|
12 |
|
|
|
|
|
13 |
This model is a medium-sized MoE implementation based on [cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser](https://huggingface.co/cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser)
|
14 |
|
15 |
-
A 2x7b configuration offers better performance than a standard 7b model even if loaded in 4 bit. (9G VRAM)
|
16 |
|
17 |
-
|
|
|
|
|
|
|
|
|
18 |
|
19 |
-
The
|
|
|
|
|
|
|
|
|
20 |
|
21 |
**These Quants will result in unpredicted behavior and I am working on new Quants as I have updated the model**
|
22 |
|
23 |
Quatizations provided by [TheBloke](https://huggingface.co/TheBloke/laser-dolphin-mixtral-2x7b-dpo-GGUF)
|
24 |
|
25 |
-
|
|
|
|
|
26 |
|
27 |
## Code Example
|
28 |
Switch the commented model definition to use in 4-bit. Should work with 9GB and still exceed the single 7B model by 5-6 points roughly
|
|
|
6 |
|
7 |

|
8 |
|
9 |
+
**New Version out now!**
|
10 |
|
11 |
Credit to Fernando Fernandes and Eric Hartford for their project [laserRMT](https://github.com/cognitivecomputations/laserRMT)
|
12 |
|
13 |
+
## Overview
|
14 |
+
|
15 |
This model is a medium-sized MoE implementation based on [cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser](https://huggingface.co/cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser)
|
16 |
|
|
|
17 |
|
18 |
+
## Process
|
19 |
+
|
20 |
+
+ The process is outlined in this [notebook](https://github.com/cognitivecomputations/laserRMT/blob/main/examples/laser-dolphin-mixtral-2x7b.ipynb)
|
21 |
+
|
22 |
+
+ The mergekit_config is in the files.
|
23 |
|
24 |
+
+ The models used in the configuration are not lasered, but the final product is. This is an update from the last version.
|
25 |
+
|
26 |
+
+ This process is experimental. Your mileage may vary.
|
27 |
+
|
28 |
+
## Quantizations
|
29 |
|
30 |
**These Quants will result in unpredicted behavior and I am working on new Quants as I have updated the model**
|
31 |
|
32 |
Quatizations provided by [TheBloke](https://huggingface.co/TheBloke/laser-dolphin-mixtral-2x7b-dpo-GGUF)
|
33 |
|
34 |
+
*Current [Quantizations](https://huggingface.co/macadeliccc/laser-dolphin-mixtral-2x7b-GGUF)*
|
35 |
+
- Q4_K_M
|
36 |
+
- Q5_K_M
|
37 |
|
38 |
## Code Example
|
39 |
Switch the commented model definition to use in 4-bit. Should work with 9GB and still exceed the single 7B model by 5-6 points roughly
|