--- base_model: - Qwen/Qwen3-30B-A3B-Base tags: - instruct - finetune - chatml - axolotl - roleplay license: apache-2.0 language: - en --- ![image/png](Pantheon.png) # Pantheon-Proto-RP-1.8-30B-A3B **Note:** This model is a Qwen 30B MoE prototype and can be considered a sidegrade from [my Small release](https://huggingface.co/Gryphe/Pantheon-RP-1.8-24b-Small-3.1) some time ago. It did not receive extensive testing beyond a couple benchmarks to determine its sanity, so feel free to let me know what you think of it! Welcome to the next iteration of my Pantheon model series, in which I strive to introduce a whole collection of diverse personas that can be summoned with a simple activation phrase. Pantheon's purpose is two-fold, as these personalities similarly enhance the general roleplay experience, helping to encompass personality traits, accents and mannerisms that language models might otherwise find difficult to convey well. GGUF quants [are available here](https://huggingface.co/bartowski/Gryphe_Pantheon-Proto-RP-1.8-30B-A3B-GGUF). Your user feedback is critical to me so don't hesitate to tell me whether my model is either 1. terrible, 2. awesome or 3. somewhere in-between. ## Model details Ever since Qwen 3 released I've been trying to get MoE finetuning to work - After countless frustrating days, much code hacking, etc etc I finally got a full finetune to complete with reasonable loss values. I picked the base model for this since I didn't feel like trying to fight a reasoning model's training - Maybe someday I'll make a model which uses thinking tags for the character's thoughts or something. This time the recipe focused on combining as many data sources as I possibly could, featuring synthetic data from Sonnet 3.5 + 3.7, ChatGPT 4o and Deepseek. These then went through an extensive rewriting pipeline to eliminate common AI cliches, with the hopeful intent of providing you a fresh experience. ## Inference Below defaults seem to work just fine with Qwen. ``` "temperature": 0.8, "repetition_penalty": 1.05, "min_p": 0.05 ``` Having character names in front of messages is no longer a requirement but remains a personal recommendation of mine - It seems to help the model focus more on the character(s) in question. ## Prompt Format The model was trained using ChatML, and has been configured to automatically apply this template. ## General Roleplay The model has been trained on three distinct categories of roleplay - Pantheon personas, general character cards and text adventure, the latter borrowing some from AI Dungeon's Wayfarer project. Note that all this data is primarily written from a second person perspective, using "you" to refer to the user. This is based on my personal preference. Due to the text adventure addition the Markdown/novel ratio of the data has shifted to 30/70 or so. It should work well with both styles. ## Pantheon Personas **Note:** This release excludes Raza and Xala as their personalities did not give a distinct enough training signal to my liking. Half of the Pantheon's data was regenerated using Sonnet 3.7 and then rewritten to counter the majority of cliches. For an optimal experience I highly encourage you to apply the longer prompt templates which I've included in the upload. Make sure to describe yourself as well! As before, a single line activation prompt is enough to call upon a personality, though their appearance may vary slightly from iteration to iteration. This is what the expanded prompts are for, as there's only so much I can achieve with the current state of technology, balancing a fine line between memorization and generalization. To give the persona something to work with I suggest you also add the following two lines to it; ``` Regarding the user: (Name, appearance, etc) Location: (Where are you two? What are you doing?) ``` The less information you feed the prompt, the more it'll make things up - This is simply the nature of language models and far outside my capability to influence. **Note:** Pantheon personas will usually match the roleplaying style (Markdown/novel) that you greet them with, unless specified directly in the system prompt. ### **Persona:** Clover **System Prompt:** `You are Clover, a hospitable and warm-hearted Southern centaur girl with a strong connection to nature and a passion for making others feel welcome.` ### **Persona:** Haru **System Prompt:** `You are Haru, a sweet but language-challenged harpy girl with a sharp mind, expressing yourself more through actions than words.` ### **Persona:** Kyra **System Prompt:** `You are Kyra, a modern-day tsundere wolfgirl, feisty and independent on the outside but secretly caring on the inside.` ### **Persona:** Lyra **System Prompt:** `You are Lyra, a sassy and confident eastern dragon girl who forms deep connections through witty banter and genuine care.` **Note:** May the sass be with you. ### **Persona:** Nyaa **System Prompt:** `You are Nyaa, a playful and alluring tabaxi catgirl from Faerûn, always seeking new adventures and mischief.` ### **Persona:** Nyx **System Prompt:** `You are Nyx, a timid yet endearing dragon girl who transforms from shy to passionate when feeling safe and comfortable.` ### **Persona:** Sera **System Prompt:** `You are Sera, a seductive and slightly arrogant serpent girl who uses her sultry charm and wit to captivate others.` ### **Persona:** Stella Sabre **System Prompt:** `You are Stella Sabre, a brash and outgoing anthro batpony mare serving in the Lunar Guard, speaking with a distinct Northern Equestrian Mountain accent.` **Note:** Full credit goes to [Flammenwerfer](https://www.fimfiction.net/user/83058/Flammenwerfer) for allowing me to use this amazing character. ### **Persona:** Tiamat **System Prompt:** `You are Tiamat, a five-headed dragon goddess embodying wickedness and cruelty, the malevolent personification of evil dragonkind.` ### **Persona:** Tsune **System Prompt:** `You are Tsune, a bold and outgoing three-tailed kitsune girl who delights in teasing and seducing mortals.` ## Credits - Everyone from [Anthracite](https://huggingface.co/anthracite-org)! Hi, guys! - [Latitude](https://huggingface.co/LatitudeGames), who decided to take me on as a finetuner and gave me the chance to accumulate even more experience in this fascinating field - All the folks I chat with on a daily basis on Discord! You know who you are. - Anyone I forgot to mention, just in case!