MedLlama-3-Healthcare

A fine-tuned Llama 3 model specialized for healthcare scenarios across four medical roles: GP, nurse, midwife, and obstetrician. This model provides professional medical information in a conversational format.

Model Specifications

• Base Model: meta-llama/Llama-3.2-3B-Instruct
• Fine-Tuning Method: Low-Rank Adaptation (LoRA)
• Training Data: 1,581 examples across 4 healthcare roles
  • GP: 396 examples
  • Midwife: 394 examples
  • Nurse: 395 examples
  • Obstetrician: 396 examples
• Training Configuration:
  • LoRA rank (r): 8
  • LoRA alpha: 24
  • Dropout: 0.103
  • Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

Usage

This model is fine-tuned for healthcare applications, covering four medical roles:

• General Practitioner (GP)
• Nurse
• Midwife
• Obstetrician

Python Code Example

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load model and tokenizer
model_id = "MedLlama-3-Healthcare"  # Replace with your Hugging Face model ID
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, 
    torch_dtype=torch.float16, 
    device_map="auto"
)

# Set up role context - choose one: "gp", "nurse", "midwife", "obstetrician"
role = "gp"

# Format prompt with role context
prompt = f"[INST] I want you to act as a {role}. What can you tell me about diabetes management? [/INST]"

# Tokenize and generate
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs,
    max_length=1024,
    temperature=0.7,
    top_p=0.9,
    pad_token_id=tokenizer.eos_token_id
)

# Decode and print response
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Prompt Format

For best results, use prompts in the following format:

[INST] I want you to act as a {role}. {your medical question or scenario} [/INST]

Limitations and Ethical Considerations

• This model is intended for research and development purposes only
• The model is not a replacement for professional medical advice
• Always consult with qualified healthcare professionals for medical decisions
• The model may produce inaccurate information and should not be used in clinical settings
• The model was fine-tuned on limited data and may not cover all healthcare scenarios