---
license: llama3.2
base_model:
- huihui-ai/Llama-3.2-3B-Instruct-abliterated
---
Using the SpinQuant quantization method from https://github.com/facebookresearch/SpinQuant, I quantized the Llama-3.2-3B-Instruct-abliterated model from https://huggingface.co/huihui-ai/Llama-3.2-3B-Instruct-abliterated. 

This quantization is for on-device deployment to Android apps with Executorch. 

To make it easier for everyone to quickly test and deploy the Executorch on-device demo, I've also converted the quantized PTH file to PTE format and uploaded it.

2025-02-16 19:40:24,099 - spinquant - INFO - wiki2 ppl is: 11.502239227294922