c4tdr0ut/NipponNative

# 🗾 nipponnative-llama3.1-8b-qlora-finetuned

Model: nipponnative/llama3.1-8b-qlora
Base: LLaMA 3.1 8B
Fine‑tuned on: 5 000 high‑quality samples (domain‑adapted)
Method: QLoRA (4‑bit adapters)
Hardware: Single NVIDIA RTX 3090 (24 GB)
Language: Japanese‑centric (multilingual support)

---

🚀 Model Overview

nipponnative-llama3.1-8b-qlora is a compact, Japanese‑oriented language model derived from Meta’s LLaMA 3.1 8B. We applied QLoRA fine‑tuning on a carefully curated 5 000-sample dataset to boost performance on Japanese language tasks while maintaining strong multilingual capabilities.

Key features:

• ⚡️ Efficient 4‑bit QLoRA adapters, minimal memory footprint.
• 📚 5 k high‑quality, domain‑specific examples (conversations, QA, creative text).
• 🗣️ High Japanese fluency, formal/informal style switching.
• 🔄 Backward compatible with Hugging Face Transformers & BitsAndBytes.

---

📊 Training Details

Attribute	Details
Base model	facebook/llama-3.1-8b
Fine‑tuning method	QLoRA (4‑bit quantized adapters)
Optimizer	AdamW (β1=0.9, β2=0.95)
Learning rate	2e-5 (linear warmup 100 steps)
Batch size (per GPU)	4
Sequence length	2048
Adapter rank (r)	16
LoRA α	32
Training steps	~1 000
Dataset size	5 000 samples (cleaned, deduped)
Hardware	Single NVIDIA RTX 3090 (24 GB VRAM)

---

💾 Dataset

Our dataset is composed of 5 000 high‑quality samples in Japanese (with some bilingual English contexts). Sources include:

• Conversational transcripts
• Technical Q&A pairs
• Creative writing prompts & completions
• Code snippets & explanations

All data was manually cleaned, deduplicated, and balanced across formality registers.

---

🔧 Installation

# 1. Clone your repo or install via pip
pip install transformers accelerate bitsandbytes

# 2. (Optional) Hugging Face CLI for private models
pip install huggingface_hub
huggingface-cli login