Update README.md
Browse files
README.md
CHANGED
@@ -14,93 +14,3 @@ datasets:
|
|
14 |
- custom-legal-dataset-pakistan
|
15 |
inference: true
|
16 |
---
|
17 |
-
|
18 |
-
# π§ LLaMA 3.2 3B Instruct β Legal QA (Pakistan Law) β Fine-Tuned LoRA
|
19 |
-
|
20 |
-
This model is a fine-tuned version of **Unsloth's LLaMA 3.2 3B Instruct** on a custom dataset of Pakistani laws, including:
|
21 |
-
|
22 |
-
- π Family Law
|
23 |
-
- π Property Law
|
24 |
-
- βοΈ Criminal Law
|
25 |
-
|
26 |
-
It is optimized using **LoRA** (Low-Rank Adaptation) with `unsloth`, making it highly efficient for legal question answering and chatbot use cases within the context of Pakistani law.
|
27 |
-
|
28 |
-
---
|
29 |
-
|
30 |
-
## π Dataset
|
31 |
-
|
32 |
-
The model was trained on a custom dataset created by parsing official PDFs of Pakistani legal acts. The structure followed a conversational format like:
|
33 |
-
|
34 |
-
\`\`\`text
|
35 |
-
### Question:
|
36 |
-
What is the procedure of talaq under Pakistani law?
|
37 |
-
|
38 |
-
### Answer:
|
39 |
-
Under Pakistani law, talaq (divorce) must be initiated by the husband in writing and sent to the relevant Union Council...
|
40 |
-
\`\`\`
|
41 |
-
|
42 |
-
---
|
43 |
-
|
44 |
-
## π οΈ How to Use
|
45 |
-
|
46 |
-
> Requires `unsloth` + `transformers`.
|
47 |
-
|
48 |
-
\`\`\`python
|
49 |
-
from unsloth import FastLanguageModel
|
50 |
-
|
51 |
-
model, tokenizer = FastLanguageModel.from_pretrained(
|
52 |
-
model_name = "heyIamUmair/llama3-3b-instruct-legal-pakistan",
|
53 |
-
max_seq_length = 2048,
|
54 |
-
dtype = None,
|
55 |
-
load_in_4bit = True,
|
56 |
-
)
|
57 |
-
|
58 |
-
FastLanguageModel.for_inference(model)
|
59 |
-
|
60 |
-
messages = [{"role": "user", "content": "What are the rights of women after divorce in Pakistani law?"}]
|
61 |
-
inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt").to("cuda")
|
62 |
-
|
63 |
-
outputs = model.generate(**inputs, max_new_tokens=200)
|
64 |
-
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
|
65 |
-
print(response)
|
66 |
-
\`\`\`
|
67 |
-
|
68 |
-
---
|
69 |
-
|
70 |
-
## π§ Technical Details
|
71 |
-
|
72 |
-
| Setting | Value |
|
73 |
-
|--------|-------|
|
74 |
-
| Base Model | `unsloth/Llama-3.2-3B-Instruct` |
|
75 |
-
| LoRA Rank | 16 |
|
76 |
-
| Optimizer | `adamw_8bit` |
|
77 |
-
| Quantization | 4-bit |
|
78 |
-
| Sequence Length | 2048 |
|
79 |
-
| Batch Size | 7 |
|
80 |
-
| Epochs | 3 |
|
81 |
-
|
82 |
-
---
|
83 |
-
|
84 |
-
## π Performance Metrics (On Test Set)
|
85 |
-
|
86 |
-
| Metric | Score |
|
87 |
-
|------------|---------|
|
88 |
-
| ROUGE-1 | `0.XX` |
|
89 |
-
| BLEU | `0.XX` |
|
90 |
-
| METEOR | `0.XX` |
|
91 |
-
|
92 |
-
_(Replace scores with your actual numbers)_
|
93 |
-
|
94 |
-
---
|
95 |
-
|
96 |
-
## π Author
|
97 |
-
|
98 |
-
This model was fine-tuned by **Umair** as part of a legal AI project focused on improving access to Pakistani legal information via LLMs.
|
99 |
-
|
100 |
-
---
|
101 |
-
|
102 |
-
## π License
|
103 |
-
|
104 |
-
Apache 2.0 β free to use, modify, and distribute.
|
105 |
-
|
106 |
-
---
|
|
|
14 |
- custom-legal-dataset-pakistan
|
15 |
inference: true
|
16 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|