how to run this model on A10 or V100

#13

by rockcat-miao - opened Jun 30, 2024

Jun 30, 2024

this model has 10 heads, so only 2 shards will working well, but... OOM when run it on A10 and V100 with 2 shards. is this mean 10 heads dislike OLD GPUs?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment