ValueError: Model architectures ['PixtralForConditionalGeneration'] failed to be inspected.
#64
by
sangwonjung
- opened
Hi, I encountered the error while running the offline code provided in the README.md
llm = LLM(model_name,
tokenizer_mode="mistral",
config_foramt="mistral",
load_format="mistral",
dtype="bfloat16",
trust_remote_code=True
)
python version = 3.11.9
vllm version = 0.8.5.dev32+g44fa4d556
transformers = 4.51.3
mistral_common = 1.5.4
I'm trying to build a batch program for text summarization using this model. How can I resolve this error?