
AV LLMs
A collection of Audio, Video and Visual LLMs.
- Text-to-Speech • Updated • 439
- 1.04k
OpenVoice
🤗 dataautogpt3/ProteusV0.3
Text-to-Image • Updated • 133k • • 93ByteDance/SDXL-Lightning
Text-to-Image • Updated • 128k • • 2kopenai/whisper-large-v3
Automatic Speech Recognition • Updated • 4.32M • • 4.17kstabilityai/TripoSR
Image-to-3D • Updated • 35.5k • 530Efficient-Large-Model/VILA-7b
Text Generation • Updated • 270 • 26google/paligemma-3b-pt-896
Image-Text-to-Text • Updated • 128k • 117microsoft/Phi-3-vision-128k-instruct
Text Generation • Updated • 80.5k • 957stabilityai/stable-audio-open-1.0
Text-to-Audio • Updated • 28.4k • 1.11kOpenVLA: An Open-Source Vision-Language-Action Model
Paper • 2406.09246 • Published • 37aiola/whisper-medusa-v1
Updated • 77 • 178merve/idefics3llama-vqav2
Updated • 8black-forest-labs/FLUX.1-schnell
Text-to-Image • Updated • 1.56M • • 3.53k- 113
Llama3.1 S V0.2 Checkpoint 2024 08 20
😻Convert text to audio and vice versa
gpt-omni/mini-omni
Text-to-Speech • Updated • 424fishaudio/fish-speech-1.4
Text-to-Speech • Updated • 1.26k • 451- 171
Tonic's GOT OCR
📲GOT - OCR (from : UCAS, Beijing)
stepfun-ai/GOT-OCR2_0
Image-Text-to-Text • Updated • 70.5k • 1.42kapple/coreml-sam2-large
Mask Generation • Updated • 47 • 25coreml-projects/sam-2-studio
Updated • 23mistralai/Pixtral-12B-2409
Image-Text-to-Text • Updated • • 624allenai/Molmo-72B-0924
Image-Text-to-Text • Updated • 4.17k • 283openai/whisper-large-v3-turbo
Automatic Speech Recognition • Updated • 7.35M • • 2.13kRevai/reverb-asr
Automatic Speech Recognition • Updated • 9 • 83- 358
GOT Online
💬Extract text from images using various OCR modes
facebook/vfusion3d
Image-to-3D • Updated • 113 • 66facebook/cotracker
Updated • 705 • 35rhymes-ai/Aria
Image-Text-to-Text • Updated • 20.8k • 620SWivid/F5-TTS
Text-to-Speech • Updated • 963k • 946- 65
Ichigo Llama3.1 S Instruct
🏢Generate text from audio recordings
kyutai/moshiko-mlx-q4
Updated • 781 • 28kyutai/moshiko-mlx-q8
Updated • 152 • 5- 101
Open VLM Video Leaderboard
🌎VLMEvalKit Eval Results in video understanding benchmark
jimmycarter/LibreFLUX
Text-to-Image • Updated • 90 • 161microsoft/OmniParser
Image-Text-to-Text • Updated • 2.45k • 1.64k- 301
Aya Models
🌍Interact with the Aya family of models.
CohereForAI/aya-expanse-32b
Text Generation • Updated • 33.4k • 238stabilityai/stable-diffusion-3.5-medium
Text-to-Image • Updated • 149k • • 647OuteAI/OuteTTS-0.1-350M
Text-to-Speech • Updated • 3.42k • 300vidore/colpali
Visual Document Retrieval • Updated • 12.5k • 424vidore/colpali-v1.2
Visual Document Retrieval • Updated • 73.2k • 106si-pbc/hertz-dev
Audio-to-Audio • Updated • 211- 38
Talk To Ultravox
⚡Talk to Fixie.ai's Ultravox with WebRTC ⚡️
LLaVA-o1: Let Vision Language Models Reason Step-by-Step
Paper • 2411.10440 • Published • 115Xkev/Llama-3.2V-11B-cot
Image-Text-to-Text • Updated • 4.79k • 147google/paligemma-3b-pt-224
Image-Text-to-Text • Updated • 23.5k • 305apple/coreml-mobileclip
Updated • 279 • 41InstantX/InstantIR
Image-to-Image • Updated • 1 • 167- 82
InstantIR
🖼diffusion-based Image Restoration model
- 143
Flux IP Adapter
🖼Prompt with Images in flux[dev]
- 38
Image Preferences - Argilla annotation space
🖼A community project to create an image preferences dataset.
fishaudio/fish-speech-1.5
Text-to-Speech • Updated • 13.7k • 507meta-llama/Llama-3.3-70B-Instruct
Text Generation • Updated • 935k • • 2.16k- 47
Paligemma2 Vqav2
🐨PaliGemma2 LoRA finetuned on VQAv2
VisionZip: Longer is Better but Not Necessary in Vision Language Models
Paper • 2412.04467 • Published • 107fancyfeast/llama-joycaption-alpha-two-hf-llava
Updated • 26.3k • 159taohu/mask
Updated • 5[MASK] is All You Need
Paper • 2412.06787 • Published • 3- 670
Open VLM Leaderboard
🌎VLMEvalKit Evaluation Results Collection
microsoft/LLM2CLIP-Llama3.2-1B-EVA02-L-14-336
Zero-Shot Image Classification • Updated • 10LLM2CLIP: Powerful Language Model Unlock Richer Visual Representation
Paper • 2411.04997 • Published • 37Generative Powers of Ten
Paper • 2312.02149 • Published • 8- 62
StoryStar
💬Fantasy story generator
GoodiesHere/Apollo-LMMs-Apollo-7B-t32
Video-Text-to-Text • Updated • 465 • 54Apollo: An Exploration of Video Understanding in Large Multimodal Models
Paper • 2412.10360 • Published • 140Qwen/Qwen2-VL-7B-Instruct
Image-Text-to-Text • Updated • 1.24M • • 1.16kXiaoduoAILab/Xmodel_VLM
Text Generation • Updated • 277 • 12nvidia/Cosmos-1.0-Diffusion-14B-Text2World
Updated • 84k • 51nvidia/Cosmos-1.0-Autoregressive-12B
Updated • 392 • 29nvidia/Cosmos-1.0-Autoregressive-13B-Video2World
Updated • 431 • 31nvidia/Cosmos-1.0-Diffusion-7B-Text2World
Updated • 143k • 210nvidia/Cosmos-1.0-Diffusion-14B-Video2World
Updated • 59k • 52- 396
Stable Point-Aware 3D
⚡Create 3D models from images
hexgrad/Kokoro-82M
Text-to-Speech • Updated • 1.58M • 3.7k- 2.28k
Kokoro TTS
❤Upgraded to v1.0!
openbmb/MiniCPM-o-2_6
Any-to-Any • Updated • 265k • 1.05k- 316
TTS Spaces Arena
🤗Blind vote on HF TTS models!
google/paligemma2-10b-pt-896
Image-Text-to-Text • Updated • 957 • 31NovaSky-AI/Sky-T1-32B-Preview
Text Generation • Updated • 8.8k • 540MiniMaxAI/MiniMax-VL-01
Image-Text-to-Text • Updated • 280 • 247- 53
SmolVLM
📊Generate descriptions from images and text prompts
HKUSTAudio/Llasa-3B
Text-to-Speech • Updated • 3.21k • 472HuggingFaceTB/SmolVLM-500M-Instruct
Image-Text-to-Text • Updated • 18k • 115deepseek-ai/Janus-Pro-7B
Any-to-Any • Updated • 248k • 3.23k- 253
Kokoro TTS Zero
🎴✨[With v1.0.0] Accelerated TTS on Kokoro-82M
kyutai/hibiki-2b-mlx-bf16
Translation • Updated • 161 • 16kyutai/hibiki-2b-pytorch-bf16
Translation • Updated • 166 • 50ARTPARK-IISc/Vaani
Viewer • Updated • 9.72M • 2.89k • 45Zyphra/Zonos-v0.1-hybrid
Text-to-Speech • Updated • 53.8k • 1.05kZyphra/Zonos-v0.1-transformer
Text-to-Speech • Updated • 201k • 383microsoft/OmniParser-v2.0
Image-Text-to-Text • Updated • 10.4k • 1.17k- 88
Paligemma2 Mix
🌖Generate text or segment objects from an image
google/paligemma2-3b-mix-448
Image-Text-to-Text • Updated • 13.3k • 39google/paligemma2-3b-mix-224
Image-Text-to-Text • Updated • 13.3k • 23google/paligemma2-28b-mix-224
Image-Text-to-Text • Updated • 73 • 4google/paligemma2-28b-mix-448
Image-Text-to-Text • Updated • 1.01k • 25google/paligemma2-10b-mix-224
Image-Text-to-Text • Updated • 1.3k • 7google/paligemma2-10b-mix-448
Image-Text-to-Text • Updated • 16.8k • 24stepfun-ai/stepvideo-t2v
Text-to-Video • Updated • 1.91k • 417stepfun-ai/stepvideo-t2v-turbo
Updated • 85Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Model
Paper • 2502.10248 • Published • 51HuggingFaceTB/SmolVLM2-2.2B-Instruct
Image-Text-to-Text • Updated • 505k • 112nvidia/canary-1b
Automatic Speech Recognition • Updated • 48.9k • 383Wan-AI/Wan2.1-I2V-14B-720P
Image-to-Video • Updated • 75.6k • 373fastrtc/kokoro-onnx
Updated • 8- 2
Fastphone
🐠Download and run an app from a Hugging Face repository
microsoft/Phi-4-multimodal-instruct
Automatic Speech Recognition • Updated • 637k • 1.17kmicrosoft/Magma-8B
Image-Text-to-Text • Updated • 13.2k • 334- 25
Magma UI
📚Magma-8B model for UI Agents
- 486
Di♪♪Rhythm
🎶Blazingly Fast and Embarrassingly Simple Song Generation
DiffRhythm: Blazingly Fast and Embarrassingly Simple End-to-End Full-Length Song Generation with Latent Diffusion
Paper • 2503.01183 • Published • 26ASLP-lab/DiffRhythm-vae
Updated • 36ASLP-lab/DiffRhythm-base
Updated • 81 • 136Large Language Diffusion Models
Paper • 2502.09992 • Published • 103GSAI-ML/LLaDA-8B-Instruct
Text Generation • Updated • 32k • 219unsloth/gemma-3-12b-pt
Image-Text-to-Text • Updated • 733 • 3google/gemma-3-27b-it
Image-Text-to-Text • Updated • 241k • • 745sesame/csm-1b
Text-to-Speech • Updated • 2.84k • 1.11kunsloth/gemma-3-27b-it-GGUF
Image-Text-to-Text • Updated • 69.2k • 62