RuntimeError: expected m1 and m2 to have the same dtype, but got: float != c10::BFloat16
According to the example provided by your company, replacing CUDA with CPU resulted in the following error:
Both max_new_tokens (=8192) and max_length(=196) seem to have been set. max_new_tokens will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Traceback (most recent call last):
File "/root/python-env/HyperCLOVAX-SEED/inference-cpu.py", line 78, in
output_ids = model.generate(
^^^^^^^^^^^^^^^
File "/root/python-env/HyperCLOVAX-SEED/HyperCLOVAX-ENV/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/root/.cache/huggingface/modules/transformers_modules/naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B/41a52658a6b224cc5e01adea5f7da644eb1228fa/modeling_hyperclovax.py", line 1140, in generate
pred = self.language_model.generate(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/python-env/HyperCLOVAX-SEED/HyperCLOVAX-ENV/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/root/python-env/HyperCLOVAX-SEED/HyperCLOVAX-ENV/lib/python3.12/site-packages/transformers/generation/utils.py", line 2465, in generate
result = self._sample(
^^^^^^^^^^^^^
File "/root/python-env/HyperCLOVAX-SEED/HyperCLOVAX-ENV/lib/python3.12/site-packages/transformers/generation/utils.py", line 3431, in _sample
outputs = self(**model_inputs, return_dict=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/python-env/HyperCLOVAX-SEED/HyperCLOVAX-ENV/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/python-env/HyperCLOVAX-SEED/HyperCLOVAX-ENV/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/python-env/HyperCLOVAX-SEED/HyperCLOVAX-ENV/lib/python3.12/site-packages/transformers/utils/generic.py", line 965, in wrapper
output = func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/python-env/HyperCLOVAX-SEED/HyperCLOVAX-ENV/lib/python3.12/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/root/python-env/HyperCLOVAX-SEED/HyperCLOVAX-ENV/lib/python3.12/site-packages/transformers/models/llama/modeling_llama.py", line 821, in forward
outputs: BaseModelOutputWithPast = self.model(
^^^^^^^^^^^
File "/root/python-env/HyperCLOVAX-SEED/HyperCLOVAX-ENV/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/python-env/HyperCLOVAX-SEED/HyperCLOVAX-ENV/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/python-env/HyperCLOVAX-SEED/HyperCLOVAX-ENV/lib/python3.12/site-packages/transformers/utils/generic.py", line 965, in wrapper
output = func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/python-env/HyperCLOVAX-SEED/HyperCLOVAX-ENV/lib/python3.12/site-packages/transformers/models/llama/modeling_llama.py", line 571, in forward
layer_outputs = decoder_layer(
^^^^^^^^^^^^^^
File "/root/python-env/HyperCLOVAX-SEED/HyperCLOVAX-ENV/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/python-env/HyperCLOVAX-SEED/HyperCLOVAX-ENV/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/python-env/HyperCLOVAX-SEED/HyperCLOVAX-ENV/lib/python3.12/site-packages/transformers/models/llama/modeling_llama.py", line 318, in forward
hidden_states, self_attn_weights = self.self_attn(
^^^^^^^^^^^^^^^
File "/root/python-env/HyperCLOVAX-SEED/HyperCLOVAX-ENV/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/python-env/HyperCLOVAX-SEED/HyperCLOVAX-ENV/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/python-env/HyperCLOVAX-SEED/HyperCLOVAX-ENV/lib/python3.12/site-packages/transformers/models/llama/modeling_llama.py", line 252, in forward
query_states = self.q_proj(hidden_states).view(hidden_shape).transpose(1, 2)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/python-env/HyperCLOVAX-SEED/HyperCLOVAX-ENV/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/python-env/HyperCLOVAX-SEED/HyperCLOVAX-ENV/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/python-env/HyperCLOVAX-SEED/HyperCLOVAX-ENV/lib/python3.12/site-packages/torch/nn/modules/linear.py", line 125, in forward
return F.linear(input, self.weight, self.bias)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: expected m1 and m2 to have the same dtype, but got: float != c10::BFloat16
