---
license: mit
language:
- en
library_name: timm
tags:
- foundation model
- pathology
- histology
---
# Lunit ONCO FM
<p align="center">
<img src="./lunit_logo.png" width="500" height="500" />
</p>

**Lunit ONCO FM** is a foundation model for computational pathology developed by [Lunit](https://www.lunit.io/), optimized for extracting informative representations from both **H&E** and **IHC** stained whole slide images (WSIs). The model is pre-trained on **50,000 WSIs** at multiple scales, enabling strong performance across a variety of downstream tasks in oncology and histopathology.

## 📦 Repository

- Hugging Face Hub: [`jeffkang-lunit/lunit-onco-fm`](https://huggingface.co/jeffkang-lunit/lunit-onco-fm)

---

## 🧠 Model Details

- **Architecture**: `vit_huge_patch14_224`
- **Parameters**: ~632M
- **Training Data**: 50K WSIs (H&E + IHC)
- **Input Resolution**: 392x392
- **MPP Scales**: [0.1944, 0.3888, 0.7776, 1.5552]

---

## 🧪 Feature Extraction Example

```python
from huggingface_hub import login
import torch
import timm
from torchvision import transforms

# Authenticate with Hugging Face
login()

# Load the model
model = timm.create_model(
        "hf_hub:jeffkang-lunit/lunit-onco-fm",
        pretrained=True,
        act_layer=torch.nn.SiLU,
        mlp_layer=timm.layers.SwiGLUPacked,
        norm_layer=functools.partial(torch.nn.RMSNorm, eps=1e-6),
)
model.to("cuda")
model.eval()

# Image preprocessing
transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize(
        mean=(0.7401,0.6559,0.7649),
        std=(0.2115,0.2564,0.1988),
    ),
])

# Dummy input
input = torch.rand(3, 224, 224)
input = transforms.ToPILImage()(input)

# Feature extraction
with torch.inference_mode():
    features = model(transform(input).unsqueeze(0).to("cuda"))

class_token = features[:, 0]    # size: 1 x 1280
patch_tokens = features[:, 5:]  # size: 1 x 256 x 1280, tokens 1-4 are register tokens so we ignore those
```

## BibTeX entry and citation info.

```
@software{ragdollv2.1,
  author = {Mingu Kang, Jack Shi, Jonghyun Lee, Sungyoon Kim, Aisha Urooj, Jeongun Ryu, Sergio Pereira, Donggeun Yoo},
  title = {vision-ssl-squad},
  url = {https://lunit.atlassian.net/wiki/spaces/AF/pages/3592290353/VisionSSL},
  year = {2025},
}
```