DFloat11/BAGEL-7B-MoT-DF11

DFloat11 Compressed Model: ByteDance-Seed/BAGEL-7B-MoT

This model uses DFloat11 lossless compression. It's 32% smaller than the original BFloat16 model, yet produces bit-identical outputs and runs efficiently on GPUs.

📊 Performance Comparison

Metric	BAGEL-7B-MoT (BFloat16)	BAGEL-7B-MoT (DFloat11)
Model Size	29.21 GB	19.89 GB
Peak GPU Memory (1024x1024 image generation)	30.07 GB	21.76 GB
Generation Time (on an A100 GPU)	54 seconds	58 seconds

🔍 How It Works

We apply Huffman coding to the exponent bits of BFloat16 model weights, which are highly compressible. We leverage hardware-aware algorithmic designs to enable highly efficient, on-the-fly weight decompression directly on the GPU. Find out more in our research paper.

🔧 How to Use

A complete usage guide is available in our GitHub repository (forked from the official Bagel repository): https://github.com/LeanModels/Bagel-DFloat11.

📄 Learn More

• Paper: 70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float
• GitHub: https://github.com/LeanModels/DFloat11
• HuggingFace: https://huggingface.co/DFloat11