Recommended inference parameters?
#1
by
MrDevolver
- opened
So, what are the recommended inference parameters?
Because I've been trying to use this model for two days now and I couldn't figure out any parameters for inference which would fix the following issues:
- Obnoxiously long CoT
- The AI started mixing up javascript with python inside the CoT in a task related to HTML and javascript
In the rare occasion when the model finally finished reasoning and started generating actual code, the code was badly formatted and generally useless. Certainly NOT worth the wait I had to endure until it finished generating and the quality of the output was certainly NOT on the same level as QwQ-32B as you placed it in your benchmarks.