Is UD-Q5_K_XL better than Q5_K_M one?
#3
by
CHNtentes
- opened
They seem to have same file size.
They seem to have same file size.
Yes use the UD one. Inference is much faster in the UD one and much better too
It's using Dynamic 2.0: https://docs.unsloth.ai/basics/unsloth-dynamic-v2.0-ggufs
They seem to have same file size.
Yes use the UD one. Inference is much faster in the UD one and much better too
It's using Dynamic 2.0: https://docs.unsloth.ai/basics/unsloth-dynamic-v2.0-ggufs
Thanks for your reply. I can understand the quality is better, since you use higher bits for more important layers.
Could you help explain why it would be faster? It seems the doc does not talk much about it, if any.