is this W8A16 or W8A8?

by ehartford - opened about 3 hours ago

Qwen org about 3 hours ago

W8A16 is compatible with Ampere with Marlin kernel

W8A8 is only compatible with Hopper.

Which is this?

Qwen org about 1 hour ago

The quantization scheme is compatible with finegrained_fp8 in Transformers.
You should be able to run it with W8A8.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment