Not a 'thinking' model if anyone is wondering
#3
by
Mdubbya
- opened
I finished quantizing the model to 8bit in GPTQ and am putting it through paces right now. Out of the gate, I was thinking this would be a 'thinking' model but that is not the case. I share this in case anyone is wondering if you will see intermediate reasoning steps.
Would be curious how this blog post might optimize the behavior here. I'm still working on getting this up on my system, so if you do find out, please let us know!