How to import only the layers relevant for creating embeddings?

#254

by Shashwat-AI-wiz - opened 18 days ago

18 days ago

I am using Google Colab to build a RAG system using Llamaindex. I intend to use the Mixtral model as a text embedding model. However, when I am loading the model in my notebook, it fills up the entire storage available to me. Hence, I want to extract only the layers that can be used for creating text embedding vectors. Can anyone help me with this?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment