---
base_model:
  - ByteDance-Seed/BAGEL-7B-MoT
base_model_relation: quantized
pipeline_tag: any-to-any
tags:
- dfloat11
- df11
- lossless compression
- 70% size, 100% accuracy
---

# DFloat11 Compressed Model: `ByteDance-Seed/BAGEL-7B-MoT`

This model uses **DFloat11** lossless compression. It's 32% smaller than the original BFloat16 model, yet produces bit-identical outputs and runs efficiently on GPUs.

### 📊 Performance Comparison

| Metric                             | BAGEL-7B-MoT (BFloat16) | BAGEL-7B-MoT (DFloat11) |
| ---------------------------------- | ------------------------- | ------------------------- |
| Model Size                         | 29.21 GB                  | 19.89 GB                  |
| Peak GPU Memory<br>(1024x1024 image generation) | 30.07 GB                  | 21.76 GB                  |
| Generation Time<br>(on an A100 GPU)   | 54 seconds               | 58 seconds               |

### 🔍 How It Works

We apply Huffman coding to the exponent bits of BFloat16 model weights, which are highly compressible. We leverage hardware-aware algorithmic designs to enable highly efficient, on-the-fly weight decompression directly on the GPU. Find out more in our [research paper](https://arxiv.org/abs/2504.11651).

### 🔧 How to Use

A complete usage guide is available in our GitHub repository (forked from the official Bagel repository): [https://github.com/LeanModels/Bagel-DFloat11](github.com/LeanModels/Bagel-DFloat11).

### 📄 Learn More

* **Paper**: [70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float](https://arxiv.org/abs/2504.11651)
* **GitHub**: [https://github.com/LeanModels/DFloat11](https://github.com/LeanModels/DFloat11)
* **HuggingFace**: [https://huggingface.co/DFloat11](https://huggingface.co/DFloat11)