GGUF Gemma3 is EXTREMELY slow !

by thusinh1969 - opened 28 days ago

Discussion

thusinh1969

28 days ago

GGUF Gemma3 is EXTREMELY slow ! Do you know why ?

Thanks,
Steve

Mungert

Owner 28 days ago

Need more info to answer. What you using to run it, hardware / software. Which quant are you using. General info for you regarding speed . The fastest quants will probably be q4_0, slowest iq quants and bf16, but really does depend on your setup.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment