heyIamUmair commited on
Commit
a674fba
Β·
verified Β·
1 Parent(s): 4eda894

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -90
README.md CHANGED
@@ -14,93 +14,3 @@ datasets:
14
  - custom-legal-dataset-pakistan
15
  inference: true
16
  ---
17
-
18
- # 🧠 LLaMA 3.2 3B Instruct – Legal QA (Pakistan Law) – Fine-Tuned LoRA
19
-
20
- This model is a fine-tuned version of **Unsloth's LLaMA 3.2 3B Instruct** on a custom dataset of Pakistani laws, including:
21
-
22
- - πŸ“œ Family Law
23
- - 🏠 Property Law
24
- - βš–οΈ Criminal Law
25
-
26
- It is optimized using **LoRA** (Low-Rank Adaptation) with `unsloth`, making it highly efficient for legal question answering and chatbot use cases within the context of Pakistani law.
27
-
28
- ---
29
-
30
- ## πŸ“Š Dataset
31
-
32
- The model was trained on a custom dataset created by parsing official PDFs of Pakistani legal acts. The structure followed a conversational format like:
33
-
34
- \`\`\`text
35
- ### Question:
36
- What is the procedure of talaq under Pakistani law?
37
-
38
- ### Answer:
39
- Under Pakistani law, talaq (divorce) must be initiated by the husband in writing and sent to the relevant Union Council...
40
- \`\`\`
41
-
42
- ---
43
-
44
- ## πŸ› οΈ How to Use
45
-
46
- > Requires `unsloth` + `transformers`.
47
-
48
- \`\`\`python
49
- from unsloth import FastLanguageModel
50
-
51
- model, tokenizer = FastLanguageModel.from_pretrained(
52
- model_name = "heyIamUmair/llama3-3b-instruct-legal-pakistan",
53
- max_seq_length = 2048,
54
- dtype = None,
55
- load_in_4bit = True,
56
- )
57
-
58
- FastLanguageModel.for_inference(model)
59
-
60
- messages = [{"role": "user", "content": "What are the rights of women after divorce in Pakistani law?"}]
61
- inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt").to("cuda")
62
-
63
- outputs = model.generate(**inputs, max_new_tokens=200)
64
- response = tokenizer.decode(outputs[0], skip_special_tokens=True)
65
- print(response)
66
- \`\`\`
67
-
68
- ---
69
-
70
- ## πŸ”§ Technical Details
71
-
72
- | Setting | Value |
73
- |--------|-------|
74
- | Base Model | `unsloth/Llama-3.2-3B-Instruct` |
75
- | LoRA Rank | 16 |
76
- | Optimizer | `adamw_8bit` |
77
- | Quantization | 4-bit |
78
- | Sequence Length | 2048 |
79
- | Batch Size | 7 |
80
- | Epochs | 3 |
81
-
82
- ---
83
-
84
- ## πŸ“ˆ Performance Metrics (On Test Set)
85
-
86
- | Metric | Score |
87
- |------------|---------|
88
- | ROUGE-1 | `0.XX` |
89
- | BLEU | `0.XX` |
90
- | METEOR | `0.XX` |
91
-
92
- _(Replace scores with your actual numbers)_
93
-
94
- ---
95
-
96
- ## πŸ™‹ Author
97
-
98
- This model was fine-tuned by **Umair** as part of a legal AI project focused on improving access to Pakistani legal information via LLMs.
99
-
100
- ---
101
-
102
- ## πŸ“Œ License
103
-
104
- Apache 2.0 β€” free to use, modify, and distribute.
105
-
106
- ---
 
14
  - custom-legal-dataset-pakistan
15
  inference: true
16
  ---