eos_token becomes <|im_end|> after GGUF conversion with llama.cpp, and generation never terminates
#1
by
mmnga
- opened
I’d like to report an issue I encountered when converting a model to GGUF with llama.cpp: the eos_token is set to <|im_end|>, which prevents generation from terminating properly.
Updating tokenizer_config.json as follows resolves the problem:
"eos_token": "<|im_end|>",
→
"eos_token": "<｜end▁of▁sentence｜>",
After making this change, the model stops correctly at the end-of-sentence token.
Additionally, I hope this information is helpful.