Cannot apply chat template from tokenizer
#31
by
DarkLight1337
- opened
The tokenizer loaded from AutoTokenizer cannot be used to apply the chat template, which is quite unexpected.
>>> from transformers import AutoProcessor, AutoTokenizer
>>> tokenizer = AutoTokenizer.from_pretrained("mistral-community/pixtral-12b")
>>> tokenizer.chat_template # The result is None
>>> processor = AutoProcessor.from_pretrained("mistral-community/pixtral-12b")
>>> processor.chat_template # The result is correct
I have tested various versions of transformers and this happens from v4.44+ (possibly earlier), so I don't think it's a problem with Transformers library.
Hey! This is because the tokenizer config doesn't have a "chat_template" field in https://huggingface.co/mistral-community/pixtral-12b/blob/main/tokenizer_config.json. We usually do not add tokenizer templates if the model is a multimodal model, i.e. we don't expect anyone to use tokenizer and image processor separately
But in Vision LLM, like gemma3, the chat template is duplicated as the model has a Gemma3ForCausaLM class which allows users to do simple text-only inferece