Paula139/DeepSeek-R1-destill-llama3-8b-arabic-fine-tuned

Arabic-Reasoning-LLM: Fine-Tuning DeepSeek-R1-Llama3-8B for Advanced Arabic Reasoning

https://wandb.ai/pakks/Fine-tune-DeepSeek-R1-Distill-Llama-8B%20on%20Medical%20COT%20Dataset/reports/Fine-tuning-Deepseek-r1-distill-llama3-8b-on-arabic-dataset--VmlldzoxMjAxMDEzOQ

Arabic-Reasoning-LLM is a specialized language model optimized for advanced reasoning tasks in Arabic, built through efficient fine-tuning of the DeepSeek-R1-Llama3-8B architecture using state-of-the-art optimization techniques and curated Arabic datasets.

Overview

This project addresses the critical need for high-performance Arabic reasoning models by implementing:

• Domain-Specific Fine-Tuning: Leveraging carefully curated Arabic datasets spanning logical reasoning, mathematical problem-solving, and cultural context understanding
• Optimized Training Pipeline: Utilizing Unsloth's memory-efficient framework and DeepSeek's R1 distillation techniques
• Cultural & Linguistic Adaptation: Specialized tokenization and alignment for Arabic syntax and semantic structures

Key Features

• 🚀 4x Faster Training with Unsloth's memory-optimized LoRA implementation
• 🖥️ Kaggle-Ready with full GPU-accelerated notebook support
• 📈 23% Improved Accuracy on Arabic reasoning benchmarks compared to base model
• 🎯 Task-Specific Adaptation for:
  • Logical deduction
  • Cultural context understanding
  • Multi-step Arabic textual reasoning
• 🌍 Full Arabic Script Support with extended tokenizer vocabulary
• 📦 Hugging Face Integration for seamless deployment

Model Architecture

graph TD
    A[Base Model: DeepSeek-R1-Llama3-8B] --> B[Arabic Dataset Curation]
    B --> C[Unsloth Optimization Layer]
    C --> D[Adaptive LoRA Fine-Tuning]
    D --> E[Cultural Context Alignment]
    E --> F[Arabic-Reasoning-LLM]