V100上无法直接推理

by MonolithFoundation - opened Sep 6, 2024

Sep 6, 2024

如果V100上安装了flashattn2，但是实际上用不了flashatten2，代码便无法推理。
设置了config atten_implementation 为sdpa，代码不认这个还是flashattn2.

Sep 9, 2024

This comment has been hidden

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment