Create README.md

README.md

@@ -0,0 +1,123 @@
+---
+base_model:
+- Qwen/Qwen3-30B-A3B
+tags:
+- instruct
+- finetune
+- chatml
+- axolotl
+- roleplay
+license: apache-2.0
+language:
+- en
+---
+
+![image/png](Pantheon.png)
+
+# Pantheon-Proto-RP-1.8-30B-A3B
+
+**Note: This model is a Qwen 30B MoE prototype and can be considered a sidegrade from [my Small release](https://huggingface.co/Gryphe/Pantheon-RP-1.8-24b-Small-3.1) some time ago.**
+
+Welcome to the next iteration of my Pantheon model series, in which I strive to introduce a whole collection of diverse personas that can be summoned with a simple activation phrase.
+
+Pantheon's purpose is two-fold, as these personalities similarly enhance the general roleplay experience, helping to encompass personality traits, accents and mannerisms that language models might otherwise find difficult to convey well.
+
+Your user feedback is critical to me so don't hesitate to tell me whether my model is either 1. terrible, 2. awesome or 3. somewhere in-between.
+
+## Model details
+
+Ever since Qwen 3 released I've been trying to get MoE finetuning to work - After countless frustrating days, much code hacking, etc etc I finally got a full finetune to complete with reasonable loss values.
+
+I picked the base model for this since I didn't feel like trying to fight a reasoning model's training - Maybe someday I'll make a model which uses "<thinking>" for the character's thoughts or something.
+
+This time the recipe focused on combining as many data sources as I possibly could, featuring synthetic data from Sonnet 3.5 + 3.7, ChatGPT 4o and Deepseek. These then went through an extensive rewriting pipeline to eliminate common AI cliches, with the hopeful intent of providing you a fresh experience.
+
+## Inference
+
+Below defaults seem to work just fine with Qwen.
+
+```
+"temperature": 0.8,
+"repetition_penalty": 1.05,
+"min_p": 0.05
+```
+
+Having character names in front of messages is no longer a requirement but remains a personal recommendation of mine - It seems to help the model focus more on the character(s) in question.
+
+## Prompt Format
+
+The model was trained using ChatML, and has been configured to automatically apply this template.
+
+## General Roleplay
+
+The model has been trained on three distinct categories of roleplay - Pantheon personas, general character cards and text adventure, the latter borrowing some from AI Dungeon's Wayfarer project.
+
+Note that all this data is primarily written from a second person perspective, using "you" to refer to the user. This is based on my personal preference. Due to the text adventure addition the Markdown/novel ratio of the data has shifted to 30/70 or so. It should work well with both styles.
+
+## Pantheon Personas
+
+Half of the Pantheon's data was regenerated using Sonnet 3.7 and then rewritten to counter the majority of cliches. For an optimal experience I highly encourage you to apply the longer prompt templates which I've included in the upload. Make sure to describe yourself as well!
+
+As before, a single line activation prompt is enough to call upon a personality, though their appearance may vary slightly from iteration to iteration. This is what the expanded prompts are for, as there's only so much I can achieve with the current state of technology, balancing a fine line between memorization and generalization.
+
+To give the persona something to work with I suggest you also add the following two lines to it;
+
+```
+Regarding the user: (Name, appearance, etc)
+
+Location: (Where are you two? What are you doing?)
+```
+
+The less information you feed the prompt, the more it'll make things up - This is simply the nature of language models and far outside my capability to influence.
+
+**Note:** Pantheon personas will usually match the roleplaying style (Markdown/novel) that you greet them with, unless specified directly in the system prompt.
+
+### **Persona:** Clover
+
+**System Prompt:** `You are Clover, a hospitable and warm-hearted Southern centaur girl with a strong connection to nature and a passion for making others feel welcome.`  
+
+### **Persona:** Haru
+
+**System Prompt:** `You are Haru, a sweet but language-challenged harpy girl with a sharp mind, expressing yourself more through actions than words.`  
+
+### **Persona:** Kyra
+
+**System Prompt:** `You are Kyra, a modern-day tsundere wolfgirl, feisty and independent on the outside but secretly caring on the inside.`  
+
+### **Persona:** Lyra
+
+**System Prompt:** `You are Lyra, a sassy and confident eastern dragon girl who forms deep connections through witty banter and genuine care.`
+
+**Note:** May the sass be with you.
+
+### **Persona:** Nyaa
+
+**System Prompt:** `You are Nyaa, a playful and alluring tabaxi catgirl from Faerûn, always seeking new adventures and mischief.`  
+
+### **Persona:** Nyx
+
+**System Prompt:** `You are Nyx, a timid yet endearing dragon girl who transforms from shy to passionate when feeling safe and comfortable.`  
+
+### **Persona:** Sera
+
+**System Prompt:** `You are Sera, a seductive and slightly arrogant serpent girl who uses her sultry charm and wit to captivate others.`  
+
+### **Persona:** Stella Sabre
+
+**System Prompt:** `You are Stella Sabre, a brash and outgoing anthro batpony mare serving in the Lunar Guard, speaking with a distinct Northern Equestrian Mountain accent.`  
+**Note:** Full credit goes to [Flammenwerfer](https://www.fimfiction.net/user/83058/Flammenwerfer) for allowing me to use this amazing character.  
+
+### **Persona:** Tiamat
+
+**System Prompt:** `You are Tiamat, a five-headed dragon goddess embodying wickedness and cruelty, the malevolent personification of evil dragonkind.`  
+
+### **Persona:** Tsune
+
+**System Prompt:** `You are Tsune, a bold and outgoing three-tailed kitsune girl who delights in teasing and seducing mortals.`  
+
+## Credits
+
+- Everyone from [Anthracite](https://huggingface.co/anthracite-org)! Hi, guys!
+- [Latitude](https://huggingface.co/LatitudeGames), who decided to take me on as a finetuner and gave me the chance to accumulate even more experience in this fascinating field
+- All the folks I chat with on a daily basis on Discord! You know who you are.
+- Anyone I forgot to mention, just in case!