Spaces:

---

huggingface-projects

/

gemma-3-12b-it

like 92

Running on Zero

Fix for Video Processing
I found a quick fix for the CUDA memory allocator error and the NVML failure when managing GPU resources.

Thank you!

Reduced Interleaved Max / Total Frames
Works for Now 🙂

2025-03-15 15:52:11.132 | DEBUG    | __main__:process_video:120 - content=[{'type': 'text', 'text': 'Frame 0.0:'}, {'type': 'image', 'url': '/tmp/tmpi0630clt.png'}, {'type': 'text', 'text': 'Frame 0.25:'}, {'type': 'image', 'url': '/tmp/tmpvyb6re43.png'}, {'type': 'text', 'text': 'Frame 0.5:'}, {'type': 'image', 'url': '/tmp/tmphl_nj0s1.png'}, {'type': 'text', 'text': 'Frame 0.75:'}, {'type': 'image', 'url': '/tmp/tmpbt6dxllo.png'}, {'type': 'text', 'text': 'Frame 1.0:'}, {'type': 'image', 'url': '/tmp/tmpsg5z01it.png'}, {'type': 'text', 'text': 'Frame 1.25:'}, {'type': 'image', 'url': '/tmp/tmpc495x96e.png'}, {'type': 'text', 'text': 'Frame 1.5:'}, {'type': 'image', 'url': '/tmp/tmpw_8i4ysx.png'}, {'type': 'text', 'text': 'Frame 1.75:'}, {'type': 'image', 'url': '/tmp/tmpo6n59bm6.png'}]
Exception in thread Thread-11 (generate):
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/transformers/generation/utils.py", line 2250, in generate
    result = self._sample(
  File "/usr/local/lib/python3.10/site-packages/transformers/generation/utils.py", line 3238, in _sample
    outputs = self(**model_inputs, return_dict=True)
  File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/transformers/models/gemma3/modeling_gemma3.py", line 1321, in forward
    image_features = self.get_image_features(pixel_values)
  File "/usr/local/lib/python3.10/site-packages/transformers/models/gemma3/modeling_gemma3.py", line 1226, in get_image_features
    vision_outputs = self.vision_tower(pixel_values=pixel_values).last_hidden_state
  File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/transformers/models/siglip/modeling_siglip.py", line 1191, in forward
    return self.vision_model(
  File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/transformers/models/siglip/modeling_siglip.py", line 1092, in forward
    encoder_outputs = self.encoder(
  File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/transformers/models/siglip/modeling_siglip.py", line 903, in forward
    layer_outputs = encoder_layer(
  File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/transformers/models/siglip/modeling_siglip.py", line 644, in forward
    hidden_states, attn_weights = self.self_attn(
  File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/transformers/models/siglip/modeling_siglip.py", line 421, in forward
    attn_weights = nn.functional.softmax(attn_weights, dim=-1, dtype=torch.float32).to(query_states.dtype)
  File "/usr/local/lib/python3.10/site-packages/torch/nn/functional.py", line 1890, in softmax
    ret = input.softmax(dim, dtype=dtype)
RuntimeError: NVML_SUCCESS == r INTERNAL ASSERT FAILED at "../c10/cuda/CUDACachingAllocator.cpp":838, please report a bug to PyTorch.