i want the API for this model to use it on college project but can't find a way to do it
#128
by
hamza1rezg1ui
- opened
i want the API for this model to use it on college project but can't find a way to do it
The best way is to serve it with Ollama
https://ollama.com/library/falcon3
Just install Ollama (Windows, macOS, Linux), enter the command
ollama run falcon3
You can also specify which model to use and which quantization type as follows:
ollama run falcon3:7b-instruct-q8_0
This command will serve the Falcon3-7B in 8-bit format (suitable for a small GPU)
You can find all of them in the link above