Jaamie/gemma_mental_health_qlora_v2

🧠 Gemma Mental Health QLoRA v2

A fine-tuned version of google/gemma-2-9b-it for mental health diagnosis using instruction-style QLoRA tuning. This model takes in user statements and predicts the most likely mental disorder in a structured dialogue format.

---

🔧 Model Details

• Base Model: google/gemma-2-9b-it
• Fine-Tuning Method: QLoRA (4-bit quantization with bitsandbytes)
• Tokenizer: ✅ Included
• LoRA Target Modules: ["q_proj", "k_proj", "v_proj", "o_proj"]
• Sequence Format:

Output format

• User: Diagnosed Mental Disorder:

---

🧪 Use Cases

• 🧠 Mental health Q&A assistant
• 🗨️ Conversational diagnosis suggestion
• 📚 NLP research and experimentation

⚠️ Disclaimer: This model is for research and educational purposes only. It is not intended for use in real-world clinical diagnosis without medical supervision.

---

💻 How to Use

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

# Load tokenizer and base + adapter model
tokenizer = AutoTokenizer.from_pretrained("Jaamie/gemma_mental_health_qlora_v2")
base_model = AutoModelForCausalLM.from_pretrained("google/gemma-2-9b-it", device_map="auto", torch_dtype=torch.float16)
model = PeftModel.from_pretrained(base_model, "Jaamie/gemma_mental_health_qlora_v2")

# Inference example
prompt = "User: I can't sleep and my thoughts are spiraling out of control.\nDiagnosed Mental Disorder:"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

with torch.no_grad():
  outputs = model.generate(**inputs, max_new_tokens=30)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))


🏋️ Training Details
Epochs: 2

Batch Size: 4 (with gradient_accumulation_steps = 2)

Max Length: 512

Quantization: 4-bit QLoRA (NF4) with bitsandbytes

Precision: bf16


# Evaluation Results

Metric	         Score
Training Loss	 3.74
Validation Loss	 3.79
Total Examples	 ~22,000


The LLM has been trained on a sample of data from the dataset containing balanced instruction-style dataset with labeled disorders.

Mental Health Class	Sample Count
Depression	4,000
Anxiety	4,000
Suicidal Thoughts	3,000
Personality Disorder	2,000
Bipolar	2,000
Stress	2,000
Normal	5,000

# Contact
Created by Jaamie Maarsh Joy Martin

🌐 https://www.linkedin.com/in/jaamie-maarsh-joy-martin/

📧 [email protected]