Exception "general.name" occurs
New here so what is wrong?
GEMMA_MODEL_ID = "google/gemma-3-12b-it-qat-q4_0-gguf"
gguf_name = "gemma-3-12b-it-q4_0.gguf"
gemma_tokenizer = None
gemma_model = None
try:
gemma_tokenizer = AutoTokenizer.from_pretrained(GEMMA_MODEL_ID, gguf_file = gguf_name)
gemma_model = AutoModelForCausalLM.from_pretrained(
GEMMA_MODEL_ID,
gguf_file = gguf_name,
torch_dtype=compute_dtype, # Use the determined compute dtype
device_map=device # Let transformers handle device placement
# Use device_map="auto" if you have multiple GPUs or want more automation, requires accelerate
)
print(f"Gemma model '{GEMMA_MODEL_ID}' loaded successfully onto device '{device}'.")
except Exception as e:
print(f"Error loading Gemma model or tokenizer: {e}")
It outputs: Error loading Gemma model or tokenizer: 'general.name'
same
I'm facing the same issue that seems to come from the modeling_gguf_pytorch_utils.py where they are reading this to fields directly and the "general.name" key is missing in the gguf file
/usr/local/lib/python3.11/dist-packages/transformers/modeling_gguf_pytorch_utils.py in load_gguf_checkpoint(gguf_checkpoint_path, return_tensors, model_to_load)
367
368 architecture = read_field(reader, "general.architecture")[0]
--> 369 model_name = read_field(reader, "general.name")
370
371 updated_architecture = None
/usr/local/lib/python3.11/dist-packages/transformers/modeling_gguf_pytorch_utils.py in read_field(reader, field)
258
259 def read_field(reader, field):
--> 260 value = reader. Fields[field]
261 return [_gguf_parse_value(value.parts[_data_index], value.types) for _data_index in value.data]
262
KeyError: 'general.name'
:(
https://huggingface.co/stduhpf/google-gemma-3-12b-it-qat-q4_0-gguf-small
Download our version here. This has the general.name metadata and the token type fixes.
