nsadeq/ReDis-Llama

Model Card for Model ID

ReDis-Llama is trained for improved inductive reasoning performance.

Model Description

• Developed by: Nafis Sadeq
• Language(s) (NLP): English
• Finetuned from model: meta-llama/Meta-Llama-3-8B-Instruct

Model Sources [optional]

• Repository: https://github.com/NafisSadeq/reasoning-distillation
• Paper: https://arxiv.org/abs/2504.10647

How to Get Started with the Model

Follow the instructions here: https://github.com/NafisSadeq/reasoning-distillation

Training Details

Training details can be found in the paper: https://arxiv.org/abs/2504.10647

Environmental Impact

• Hardware Type: 2 × 48 GB Nvidia RTX A6000 GPUs
• Hours used: 72 hours

Model Architecture and Objective

This model has the same architecture as meta-llama/Meta-Llama-3-8B-Instruct

Compute Infrastructure

2 × 48 GB Nvidia RTX A6000 GPUs

Citation

If you use this model, please cite the following paper.

@misc{sadeq2025improvingincontextlearningreasoning, title={Improving In-Context Learning with Reasoning Distillation}, author={Nafis Sadeq and Xin Xu and Zhouhang Xie and Julian McAuley and Byungkyu Kang and Prarit Lamba and Xiang Gao}, year={2025}, eprint={2504.10647}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2504.10647}, }