Cannot apply chat template from tokenizer

#31
by DarkLight1337 - opened

The tokenizer loaded from AutoTokenizer cannot be used to apply the chat template, which is quite unexpected.

>>> from transformers import AutoProcessor, AutoTokenizer
>>> tokenizer = AutoTokenizer.from_pretrained("mistral-community/pixtral-12b")
>>> tokenizer.chat_template  # The result is None
>>> processor = AutoProcessor.from_pretrained("mistral-community/pixtral-12b")
>>> processor.chat_template  # The result is correct

I have tested various versions of transformers and this happens from v4.44+ (possibly earlier), so I don't think it's a problem with Transformers library.

Unofficial Mistral Community org

Hey! This is because the tokenizer config doesn't have a "chat_template" field in https://huggingface.co/mistral-community/pixtral-12b/blob/main/tokenizer_config.json. We usually do not add tokenizer templates if the model is a multimodal model, i.e. we don't expect anyone to use tokenizer and image processor separately

But in Vision LLM, like gemma3, the chat template is duplicated as the model has a Gemma3ForCausaLM class which allows users to do simple text-only inferece

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment