metadata
license: mit
tags:
- unsloth
- trl
- sft
ape = Algorithmic Pattern Emulation
A finetune of Mistral Nemo Instruct 2407 using my fullfictions-85kmax dataset. The prompts are fairly simple. My goal is to train a model that can write long ficiton that make sense. The training data contains the full text of public domain short stories and novels. 85k of context is about the limit I've been able to train without getting OOM errors using rented GPUs.
Thanks to unsloth brothers, gutenberg volunteers, the Mistral Nemo team, and the folks in various discord servers who have helped me out.