Update README.md
Browse files
README.md
CHANGED
@@ -4,6 +4,52 @@ license: gemma
|
|
4 |
|
5 |
# <span style="color: #7FFF7F;">Gemma-3 4B Instruct GGUF Models</span>
|
6 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
7 |
|
8 |
## **Choosing the Right Model Format**
|
9 |
|
|
|
4 |
|
5 |
# <span style="color: #7FFF7F;">Gemma-3 4B Instruct GGUF Models</span>
|
6 |
|
7 |
+
## How to Use Gemma 3 Vision with llama.cpp
|
8 |
+
|
9 |
+
To utilize the experimental support for Gemma 3 Vision in `llama.cpp`, follow these steps:
|
10 |
+
|
11 |
+
1. **Clone the lastest llama.cpp Repository**:
|
12 |
+
```bash
|
13 |
+
git clone https://github.com/ggml-org/llama.cpp.git
|
14 |
+
cd llama.cpp
|
15 |
+
```
|
16 |
+
|
17 |
+
|
18 |
+
2. **Build the Llama.cpp**:
|
19 |
+
|
20 |
+
Build llama.cpp as usual : https://github.com/ggml-org/llama.cpp#building-the-project
|
21 |
+
|
22 |
+
3. **Download the Gemma 3 gguf file**:
|
23 |
+
|
24 |
+
4. **Download the Gemma 3 mmproj file**
|
25 |
+
|
26 |
+
6. **Run the CLI Tool**:
|
27 |
+
```bash
|
28 |
+
llama-gemma3-cli -m google_gemma-3-4b-it-f16.gguf --mmproj google_gemma-3-4b-it-mmproj-f16.gguf
|
29 |
+
```
|
30 |
+
7. Copy images to the same folder as the gguf files or alter paths appropriately
|
31 |
+
|
32 |
+
|
33 |
+
Running in chat mode, available commands:
|
34 |
+
/image <path> load an image
|
35 |
+
/clear clear the chat history
|
36 |
+
/quit or /exit exit the program
|
37 |
+
|
38 |
+
```
|
39 |
+
> hi
|
40 |
+
Hello! How's it going today?
|
41 |
+
|
42 |
+
Is there something specific on your mind, or were you simply saying hi? 😊
|
43 |
+
|
44 |
+
I’m here to chat, answer questions, help with creative tasks, or just listen – whatever you need!
|
45 |
+
|
46 |
+
> /image ./bliss.png
|
47 |
+
Encoding image ./bliss.png
|
48 |
+
|
49 |
+
> what is that
|
50 |
+
That's a beautiful image!
|
51 |
+
```
|
52 |
+
|
53 |
|
54 |
## **Choosing the Right Model Format**
|
55 |
|