How to continue pt / sft on this model,any suggestions?

#53
by Ken0102030405 - opened

I wanna fit the model into domin area, how to inject my domin knowledge,any suggestions?

can I use LoRA in sft ??

If your data does not contain CoT, the fine-tuned results may disrupt CoT, causing the <think> tag to disappear. I am currently facing this issue and am unsure how to resolve it.

If your data does not contain CoT, the fine-tuned results may disrupt CoT, causing the <think> tag to disappear. I am currently facing this issue and am unsure how to resolve it.

I haven't tested on this model., while I tried to sft deepseek_distill with CoT content wrapped by tag, the result seems good.

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment