MMTEB: Massive Multilingual Text Embedding Benchmark
Paper
•
2502.13595
•
Published
•
32
Webhooks Explorers (BETA)
community
AI & ML interests
Webhooks are now publicly available on Hugging Face!
webhooks-explorers's activity
bpHigh
authored
a
paper
about 5 hours ago
Post
1387
Gemma 3 Release in a nutshell
(seems like function calling is not supported whereas the announcement said so)
(seems like function calling is not supported whereas the announcement said so)
Post
2020
I introduce MIT license
ML Model Specialize Fine Tuner app "SFT Tiny Titans" 🚀
Demo video with source.
Download, train, SFT, and test your models, easy as 1-2-3!
URL: awacke1/TorchTransformers-NLP-CV-SFT
ML Model Specialize Fine Tuner app "SFT Tiny Titans" 🚀
Demo video with source.
Download, train, SFT, and test your models, easy as 1-2-3!
URL: awacke1/TorchTransformers-NLP-CV-SFT
- 2 replies
taesiri
authored
a
paper
10 days ago
davidberenstein1957
posted
an
update
11 days ago
Post
2255
🔥 Text2SQL, explore and share any data analysis!
🤗 Hugging Face - Dataset Studio is an amazing new feature.
🚀 Start yourself: fka/awesome-chatgpt-prompts
📺 YouTube: https://youtu.be/5LUZq7MHolA?feature=shared
🤗 Hugging Face - Dataset Studio is an amazing new feature.
🚀 Start yourself: fka/awesome-chatgpt-prompts
📺 YouTube: https://youtu.be/5LUZq7MHolA?feature=shared
davidberenstein1957
posted
an
update
12 days ago
Post
4182
🥊 Epic Agent Framework Showdown! Available today!
🔵 In the blue corner, the versatile challenger with a proven track record of knowledge retrieval: LlamaIndex!
🛑 In the red corner, the defender, weighing in with lightweight efficiency: Hugging Face smolagents!
🔗 URL: https://huggingface.co/agents-course
We just published the LlamaIndex unit for the agents course, and it is set to offer a great contrast between the smolagents unit by looking at
- What makes llama-index stand-out
- How the LlamaHub is used for integrations
- Creating QueryEngine components
- Using agents and tools
- Agentic and multi-agent workflows
The team has been working flat-out on this for a few weeks. Supported by Logan Markewich and Laurie Voss over at LlamaIndex.
Who won? You decide!
🔵 In the blue corner, the versatile challenger with a proven track record of knowledge retrieval: LlamaIndex!
🛑 In the red corner, the defender, weighing in with lightweight efficiency: Hugging Face smolagents!
🔗 URL: https://huggingface.co/agents-course
We just published the LlamaIndex unit for the agents course, and it is set to offer a great contrast between the smolagents unit by looking at
- What makes llama-index stand-out
- How the LlamaHub is used for integrations
- Creating QueryEngine components
- Using agents and tools
- Agentic and multi-agent workflows
The team has been working flat-out on this for a few weeks. Supported by Logan Markewich and Laurie Voss over at LlamaIndex.
Who won? You decide!
davidberenstein1957
posted
an
update
13 days ago
Post
2990
🫸 New release to push vector search to the Hub with vicinity and work with any serialisable objects.
🧑🏫 KNN, HNSW, USEARCH, ANNOY, PYNNDESCENT, FAISS, and VOYAGER.
🔗 Example Repo: minishlab/my-vicinity-repo
🧑🏫 KNN, HNSW, USEARCH, ANNOY, PYNNDESCENT, FAISS, and VOYAGER.
🔗 Example Repo: minishlab/my-vicinity-repo
Post
2376
🚀 Blast into the future with ZaxxonGalaxian – a thrilling 3D action game where you navigate epic battles through towering 3D cityscapes! Face off against relentless swarm bots, climb the leaderboard, and dominate the skies.
awacke1/ZaxxoGalaxian
taesiri
authored
a
paper
27 days ago
davidberenstein1957
posted
an
update
about 1 month ago
Post
3290
🚀 Find banger tools for your smolagents!
I created the Tools gallery, which makes tools specifically developed by/for smolagents searchable and visible. This will help with:
- inspiration
- best practices
- finding cool tools
Space: davidberenstein1957/smolagents-and-tools
I created the Tools gallery, which makes tools specifically developed by/for smolagents searchable and visible. This will help with:
- inspiration
- best practices
- finding cool tools
Space: davidberenstein1957/smolagents-and-tools
- 1 reply
vumichien
authored
a
paper
about 1 month ago
davidberenstein1957
posted
an
update
about 1 month ago
Post
2475
Fine-tune Deepseek-R1 with a Synthetic Reasoning Dataset
Blog: https://huggingface.co/blog/sdiazlor/fine-tune-deepseek-with-a-synthetic-reasoning-data
Blog: https://huggingface.co/blog/sdiazlor/fine-tune-deepseek-with-a-synthetic-reasoning-data
davidberenstein1957
posted
an
update
about 1 month ago
Post
2073
Agentic RAG: Applied, visual, and step-by-step! 🐾
Get familiar with the Agents and tools, not the bells and whistles!
Retrieve - Augment and now GENERATE.
part 3: https://huggingface.co/blog/davidberenstein1957/ai-blueprint-agentic-rag-part-3-generate
Get familiar with the Agents and tools, not the bells and whistles!
Retrieve - Augment and now GENERATE.
part 3: https://huggingface.co/blog/davidberenstein1957/ai-blueprint-agentic-rag-part-3-generate
davidberenstein1957
posted
an
update
about 1 month ago
Post
2878
Anyone can create free hosted tools for their AI agents! 🔥
Agentic RAG stack part 2 - augment
Augment retrieval results by reranking optimises content without increasing time too much
part2: https://huggingface.co/blog/davidberenstein1957/ai-blueprint-agentic-rag-part-2-augment
code: https://github.com/huggingface/ai-blueprint
Agentic RAG stack part 2 - augment
Augment retrieval results by reranking optimises content without increasing time too much
part2: https://huggingface.co/blog/davidberenstein1957/ai-blueprint-agentic-rag-part-2-augment
code: https://github.com/huggingface/ai-blueprint
davidberenstein1957
posted
an
update
about 1 month ago
Post
2008
Creating an agentic RAG stack on the Hugging Face Hub - part 1 - retrieval (1/5).
🚀 Web apps and microservices included!
Chunk, embed and index documents at a huge scale without overhead.
Blog: https://huggingface.co/blog/davidberenstein1957/ai-blueprint-agentic-rag-part-1-retrieve
🚀 Web apps and microservices included!
Chunk, embed and index documents at a huge scale without overhead.
Blog: https://huggingface.co/blog/davidberenstein1957/ai-blueprint-agentic-rag-part-1-retrieve
Post
3007
Simple Paper Review #5
I briefly reviewed the paper "SFT Memorizes, RL Generalizes," which compares SFT and RL in post-training of LLM/VLM from HKU, UC Berkeley, Google DeepMind, and New York University
The conclusion suggests SFT excels in memorization, while RL is better for generalization. However, since LLM/VLM should benefit humans beyond just generalization, a mix of SFT and RL is advisable. Typically, some SFT is followed by RL to understand prompt formats and enhance generalization through trial and error.
The study focused on one model, Llama-3.2-Vision-11B, using environments like General Points for arithmetic reasoning and V-IRL for spatial reasoning. Training data was used for both SFT and RL, with evaluations on in-distribution and out-of-distribution data to assess memorization and generalization.
I want to apply RL extensively, but it requires building a similar simulation environment. For domain-specific models, significant investment in creating a "playground" for the model is crucial, as the effort will directly influence the outcomes.
https://arxiv.org/abs/2501.17161
I briefly reviewed the paper "SFT Memorizes, RL Generalizes," which compares SFT and RL in post-training of LLM/VLM from HKU, UC Berkeley, Google DeepMind, and New York University
The conclusion suggests SFT excels in memorization, while RL is better for generalization. However, since LLM/VLM should benefit humans beyond just generalization, a mix of SFT and RL is advisable. Typically, some SFT is followed by RL to understand prompt formats and enhance generalization through trial and error.
The study focused on one model, Llama-3.2-Vision-11B, using environments like General Points for arithmetic reasoning and V-IRL for spatial reasoning. Training data was used for both SFT and RL, with evaluations on in-distribution and out-of-distribution data to assess memorization and generalization.
I want to apply RL extensively, but it requires building a similar simulation environment. For domain-specific models, significant investment in creating a "playground" for the model is crucial, as the effort will directly influence the outcomes.
https://arxiv.org/abs/2501.17161
Post
4356
A brief summary of the o3-mini
The OpenAI o3-mini model is a significant improvement over the o1-mini, reaching o1 performance levels. While generally good, its performance isn't universally better than previous models (o1, o1-prev.) or GPT-4o across all benchmarks. This means workflows should be re-evaluated with each model upgrade.
The o3-mini has "low," "medium," and "high" versions, with "low" being the base model used for benchmarking. It's speculated that the higher versions simply involve more processing. A fair comparison with other models like Gemini 2.0 Thinking or DeepSeek-R1 would likely need to use the "low" version and a similar "think more" mechanism.
The system card is recommended reading due to its comprehensive benchmark data.
https://openai.com/index/openai-o3-mini/
The OpenAI o3-mini model is a significant improvement over the o1-mini, reaching o1 performance levels. While generally good, its performance isn't universally better than previous models (o1, o1-prev.) or GPT-4o across all benchmarks. This means workflows should be re-evaluated with each model upgrade.
The o3-mini has "low," "medium," and "high" versions, with "low" being the base model used for benchmarking. It's speculated that the higher versions simply involve more processing. A fair comparison with other models like Gemini 2.0 Thinking or DeepSeek-R1 would likely need to use the "low" version and a similar "think more" mechanism.
The system card is recommended reading due to its comprehensive benchmark data.
https://openai.com/index/openai-o3-mini/
davidberenstein1957
posted
an
update
about 2 months ago
Post
1632
tldr; Parquet is awesome, DuckDB too!
Datasets on the Hugging Face Hub rely on parquet files. We can interact with these files using DuckDB as a fast in-memory database system. One of DuckDB’s features is vector similarity search which can be used with or without an index.
blog:
https://huggingface.co/learn/cookbook/vector_search_with_hub_as_backend
Datasets on the Hugging Face Hub rely on parquet files. We can interact with these files using DuckDB as a fast in-memory database system. One of DuckDB’s features is vector similarity search which can be used with or without an index.
blog:
https://huggingface.co/learn/cookbook/vector_search_with_hub_as_backend
Post
2023
Simple summary on DeepSeek AI's Janus-Pro: A fresh take on multimodal AI!
It builds on its predecessor, Janus, by tweaking the training methodology rather than the model architecture. The result? Improved performance in understanding and generating multimodal data.
Janus-Pro uses a three-stage training strategy, similar to Janus, but with key modifications:
✦ Stage 1 & 2: Focus on separate training for specific objectives, rather than mixing data.
✦ Stage 3: Fine-tuning with a careful balance of multimodal data.
Benchmarks show Janus-Pro holds its own against specialized models like TokenFlow XL and MetaMorph, and other multimodal models like SD3 Medium and DALL-E 3.
The main limitation? Low image resolution (384x384). However, this seems like a strategic choice to focus on establishing a solid "recipe" for multimodal models. Future work will likely leverage this recipe and increased computing power to achieve higher resolutions.
It builds on its predecessor, Janus, by tweaking the training methodology rather than the model architecture. The result? Improved performance in understanding and generating multimodal data.
Janus-Pro uses a three-stage training strategy, similar to Janus, but with key modifications:
✦ Stage 1 & 2: Focus on separate training for specific objectives, rather than mixing data.
✦ Stage 3: Fine-tuning with a careful balance of multimodal data.
Benchmarks show Janus-Pro holds its own against specialized models like TokenFlow XL and MetaMorph, and other multimodal models like SD3 Medium and DALL-E 3.
The main limitation? Low image resolution (384x384). However, this seems like a strategic choice to focus on establishing a solid "recipe" for multimodal models. Future work will likely leverage this recipe and increased computing power to achieve higher resolutions.
davidberenstein1957
posted
an
update
about 2 months ago
Post
1807
Let's uncover the post-training dataset from DeepSeek-R1 with Magpie!
Pass pre-query tokens
We can get realistic examples!
Gist: https://gist.github.com/davidberenstein1957/3f20046ce57395a6aba13f8b4e956b59
Pass pre-query tokens
<｜begin▁of▁sentence｜>User: , let the model generate the rest.We can get realistic examples!
Gist: https://gist.github.com/davidberenstein1957/3f20046ce57395a6aba13f8b4e956b59
-
- 6 replies