s3nh
s3nh
AI & ML interests
Quantization, LLMs, Deep Learning for good. Follow me if you like my work. Patreon.com/s3nh
Recent Activity
liked
a model
1 day ago
nari-labs/Dia-1.6B
new activity
1 day ago
city96/Wan2.1-FLF2V-14B-720P-gguf:Pipeline
reacted
to
as-cle-bert's
post
with ❤️
about 2 months ago
I just released a fully automated evaluation framework for your RAG applications!📈
GitHub 👉 https://github.com/AstraBert/diRAGnosis
PyPi 👉 https://pypi.org/project/diragnosis/
It's called 𝐝𝐢𝐑𝐀𝐆𝐧𝐨𝐬𝐢𝐬 and is a lightweight framework that helps you 𝗱𝗶𝗮𝗴𝗻𝗼𝘀𝗲 𝘁𝗵𝗲 𝗽𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲 𝗼𝗳 𝗟𝗟𝗠𝘀 𝗮𝗻𝗱 𝗿𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 𝗺𝗼𝗱𝗲𝗹𝘀 𝗶𝗻 𝗥𝗔𝗚 𝗮𝗽𝗽𝗹𝗶𝗰𝗮𝘁𝗶𝗼𝗻𝘀.
You can launch it as an application locally (it's Docker-ready!🐋) or, if you want more flexibility, you can integrate it in your code as a python package📦
The workflow is simple:
🧠 You choose your favorite LLM provider and model (supported, for now, are Mistral AI, Groq, Anthropic, OpenAI and Cohere)
🧠 You pick the embedding models provider and the embedding model you prefer (supported, for now, are Mistral AI, Hugging Face, Cohere and OpenAI)
📄 You prepare and provide your documents
⚙️ Documents are ingested into a Qdrant vector database and transformed into a synthetic question dataset with the help of LlamaIndex
📊 The LLM is evaluated for the faithfulness and relevancy of its retrieval-augmented answer to the questions
📊 The embedding model is evaluated for hit rate and mean reciprocal ranking (MRR) of the retrieved documents
And the cool thing is that all of this is 𝗶𝗻𝘁𝘂𝗶𝘁𝗶𝘃𝗲 𝗮𝗻𝗱 𝗰𝗼𝗺𝗽𝗹𝗲𝘁𝗲𝗹𝘆 𝗮𝘂𝘁𝗼𝗺𝗮𝘁𝗲𝗱: you plug it in, and it works!🔌⚡
Even cooler? This is all built on top of LlamaIndex and its integrations: no need for tons of dependencies or fancy workarounds🦙
And if you're a UI lover, Gradio and FastAPI are there to provide you a seamless backend-to-frontend experience🕶️
So now it's your turn: you can either get diRAGnosis from GitHub 👉 https://github.com/AstraBert/diRAGnosis
or just run a quick and painless:
```bash
uv pip install diragnosis
```
To get the package installed (lightning-fast) in your environment🏃♀️
Have fun and feel free to leave feedback and feature/integrations requests on GitHub issues✨
Organizations
s3nh's activity
Adding `safetensors` variant of this model
#1 opened 3 months ago
by
SFconvertbot
Gh organization
8
#3 opened 4 months ago
by
s3nh
Datasets
3
#1 opened 4 months ago
by
s3nh
Create model card with metadata
#1 opened about 1 year ago
by
osanseviero
Create model card with metadata
#1 opened about 1 year ago
by
osanseviero
Adding Evaluation Results
#2 opened about 1 year ago
by
leaderboard-pr-bot
Adding Evaluation Results
#1 opened about 1 year ago
by
leaderboard-pr-bot
Adding Evaluation Results
#1 opened about 1 year ago
by
leaderboard-pr-bot
Adding Evaluation Results
#1 opened about 1 year ago
by
leaderboard-pr-bot
RuntimeError: CUDA must not be initialized in the main process on Spaces with Stateless GPU environment.
1
2
#6 opened about 1 year ago
by
mrfakename
error loading model: create_tensor: tensor 'output.weight' not found ?
6
#1 opened over 1 year ago
by
wukongai
Hello bro
12
#1 opened over 1 year ago
by
Abhaykoul
Librarian Bot: Add base_model metadata to model
#2 opened over 1 year ago
by
librarian-bot
Librarian Bot: Add merge tag to model
#1 opened over 1 year ago
by
librarian-bot
Problem with Quants
5
#2 opened over 1 year ago
by
Clevyby
Thanks :)
1
1
#1 opened over 1 year ago
by
BlueNipples
not able to post after exceed message length once
1
#19 opened over 1 year ago
by
s3nh
Deleted GGUF?
6
#1 opened over 1 year ago
by
EloyOn