jeffkang-lunit/lunit-onco-fm

Lunit ONCO FM

Lunit ONCO FM is a foundation model for computational pathology developed by Lunit, optimized for extracting informative representations from both H&E and IHC stained whole slide images (WSIs). The model is pre-trained on 50,000 WSIs at multiple scales, enabling strong performance across a variety of downstream tasks in oncology and histopathology.

📦 Repository

• Hugging Face Hub: jeffkang-lunit/lunit-onco-fm

---

🧠 Model Details

• Architecture: vit_huge_patch14_224
• Parameters: ~632M
• Training Data: 50K WSIs (H&E + IHC)
• Input Resolution: 392x392
• MPP Scales: [0.1944, 0.3888, 0.7776, 1.5552]

---

🧪 Feature Extraction Example

from huggingface_hub import login
import torch
import timm
from torchvision import transforms

# Authenticate with Hugging Face
login()

# Load the model
model = timm.create_model(
        "hf_hub:jeffkang-lunit/lunit-onco-fm",
        pretrained=True,
        act_layer=torch.nn.SiLU,
        mlp_layer=timm.layers.SwiGLUPacked,
        norm_layer=functools.partial(torch.nn.RMSNorm, eps=1e-6),
)
model.to("cuda")
model.eval()

# Image preprocessing
transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize(
        mean=(0.7401,0.6559,0.7649),
        std=(0.2115,0.2564,0.1988),
    ),
])

# Dummy input
input = torch.rand(3, 224, 224)
input = transforms.ToPILImage()(input)

# Feature extraction
with torch.inference_mode():
    features = model(transform(input).unsqueeze(0).to("cuda"))

class_token = features[:, 0]    # size: 1 x 1280
patch_tokens = features[:, 5:]  # size: 1 x 256 x 1280, tokens 1-4 are register tokens so we ignore those

BibTeX entry and citation info.

@software{ragdollv2.1,
  author = {Mingu Kang, Jack Shi, Jonghyun Lee, Sungyoon Kim, Aisha Urooj, Jeongun Ryu, Sergio Pereira, Donggeun Yoo},
  title = {vision-ssl-squad},
  url = {https://lunit.atlassian.net/wiki/spaces/AF/pages/3592290353/VisionSSL},
  year = {2025},
}