---
license: mit
language:
- zho
- eng
- fra
- spa
- por
- deu
- ita
- rus
- jpn
- kor
- vie
- tha
- ara
base_model:
- Qwen/Qwen2.5-0.5B-Instruct
pipeline_tag: text-generation
---

![image/png](https://cdn-uploads.huggingface.co/production/uploads/667e7043f6fa6fb03b220f9c/lYY8N3OPJo6yLTay8g0qN.png)

# Zeta 1 - First Step of Zeta Project, This is Epoch.
Zeta 1 is the first step that I, an ordinary person and consumer, have taken in pursuit of my dream as an individual.

## About Zeta 1
Zeta 1 is an LLM with about 400 million parameters.

It might be better to call it an SLM.

Zeta 1 is an SLM that was painstakingly created on a consumer computer.

### Computer Spec

**Machine:** *Mac Mini (M2 Pro, 32GB RAM, 2023)*

This is a technical matter, but on a Mac with Apple Silicon you can only choose between CPU or MPS (Apple's proprietary API).

CPU has some compatibility but is too slow.

MPS is somewhat faster but doesn't allow the use of optimization systems such as fp16.

Zeta 1 was built using MPS.

### Arguments of Trainer
- **train epochs:** 3
- **warmup steps:** 100

### Datasets
Details of the dataset used can be found [here](https://github.com/DiamondGotCat/Zeta-Dataset/releases/tag/zeta-1)

## Links
GitHub: [Zeta](https://github.com/DiamondGotCat/Zeta)

---

Zeta is just a small SLM.
But don't forget that it has big dreams inside.