Memory requirement for 13B 4Bit

by afoam - opened Apr 15, 2023

afoam

Apr 15, 2023

I'm new to this, how much RAM does the 13b 4 bit quantized model require? Will it run on 32 GB? Thank you

Owner Apr 15, 2023

~11GB, but some headroom is nice. A 16GB system runs the 13B model perfectly.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment