observerw/ChiseLLM-7B

ChiseLLM Models

ChiseLLM is a series of large reasoning models specifically trained for the Chisel Hardware Construction language, aimed at revolutionizing HCL-Baed Agile Hardware Development Methodology (AHDM).

Built on Qwen/Qwen2.5-Coder-Instruct with domain-adaptive fine-tuning, the model combines high-quality reasoning datasets and specific thinking patterns to significantly enhance performance in hardware design tasks.

ChiseLLM Models can:

• Transform natural language specifications into high-quality Chisel code (Spec-to-Chisel)
• Intelligently translate Verilog code into enhanced Chisel implementations (Decompile-to-Chisel)
• Generate hardware designs with superior variability and extensibility, surpassing traditional design approaches

Use Cases

ChiseLLM Models is particularly suited for the following applications:

• Rapid Hardware Design Prototyping: Dramatically shortens the design cycle from specification to implementation
• Verilog Code Modernization: Intelligently converts legacy Verilog code into extensible Chisel implementations
• Hardware Architecture Exploration: Generates multiple design variants for the same functional requirements
• Design Refactoring and Optimization: Leverages Chisel's advanced features to improve existing hardware designs
• Agile Hardware Development Education: Serves as an assistive tool for learning Chisel and modern hardware design methods

Training results

Spec-to-Chisel task on VerilogEval:

Models	pass@1	pass@3	pass@5	syntax(%)
Llama3.1-8B-Instruct	4.33	9.90	13.21	9.02
Qwen2.5-Coder-7B-Instruct	21.94	31.87	36.73	37.08
*Deepseek-R1-Distill-Llama-8B	9.31	15.44	17.72	16.01
*ChiseLLM-7B	29.41	47.08	54.04	58.82

Models	pass@1	pass@3	pass@5	syntax(%)
Qwen2.5-Coder-32B-Instruct	41.02	53.85	58.79	73.47
Qwen2.5-72B-Instruct	39.74	49.30	52.90	61.31
Llama-3.3-70B-Instruct	38.14	44.90	48.02	65.97
*Deepseek-R1-Distill-Qwen-32B	38.50	54.58	61.16	52.19
*Deepseek-R1-Distill-Llama-70B	36.62	52.28	59.90	51.72
*ChiseLLM-32B	51.43	68.29	72.78	76.45

Models	pass@1	pass@3	pass@5	syntax(%)
Deepseek-V3	50.16	63.44	67.32	76.37
GPT-4o	42.04	60.16	65.17	69.76
*Deepseek-R1	62.74	76.05	80.16	82.85

Decompile-to-Chisel task on VerilogEval:

Models	pass@1	pass@3	pass@5	syntax(%)
Llama3.1-8B-Instruct	5.43	12.29	16.08	11.15
Qwen2.5-Coder-7B-Instruct	27.60	34.58	37.19	43.23
*Deepseek-R1-Distill-Llama-8B	10.05	16.15	18.13	12.03
* ChiseLLM-7B	50.47	70.99	78.08	59.19

Models	pass@1	pass@3	pass@5	syntax(%)
Qwen2.5-Coder-32B-Instruct	41.19	48.96	51.59	53.93
Qwen2.5-72B-Instruct	40.54	47.32	49.83	59.30
Llama-3.3-70B-Instruct	38.38	46.96	51.36	48.00
*Deepseek-R1-Distill-Qwen-32B	45.03	63.02	70.18	53.17
*Deepseek-R1-Distill-Llama-70B	37.50	55.05	63.84	45.59
*ChiseLLM-32B	56.41	72.00	77.67	64.71

Models	pass@1	pass@3	pass@5	syntax(%)
Deepseek-V3	54.57	63.19	66.71	66.19
GPT-4o	42.39	65.75	71.83	53.77
*Deepseek-R1	53.45	71.50	77.91	59.13

Framework versions

• Transformers 4.51.0
• Pytorch 2.6.0a0+df5bbc09d1.nv24.12
• Datasets 3.4.1
• Tokenizers 0.21.0

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 2
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 8
• gradient_accumulation_steps: 8
• total_train_batch_size: 128
• total_eval_batch_size: 64
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.05
• num_epochs: 3.0

Citation

If you are interested in our work, please consider citing this, it would be greatly appreciated!

@misc{wang2025chisellmunleashingpowerreasoning,
      title={ChiseLLM: Unleashing the Power of Reasoning LLMs for Chisel Agile Hardware Development}, 
      author={Bowei Wang and Jiaran Gao and Yelai Feng and Renzhi Chen and Shanshan Li and Lei Wang},
      year={2025},
      eprint={2504.19144},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2504.19144}, 
}