ajr0/mistral-7b-instruct-v0.3-bnb-4bit-hon-1k-v1.3

Mistral-7B Instruct v0.3 - Fine-tuned on HumanOrNot Chats

• Developed by: ajr0
• License: apache-2.0
• Finetuned from model: unsloth/mistral-7b-instruct-v0.3-bnb-4bit
• Dataset: Curated subset of humanornot.ai chat logs.

Model Description

This repository contains a fine-tuned version of unsloth/mistral-7b-instruct-v0.3-bnb-4bit. The model was specifically fine-tuned on a curated dataset of 1,000 chat conversations sourced from humanornot.ai. The goal of this fine-tuning was to explore the adaptation of the Mistral Instruct model to the specific conversational patterns and styles present in the HumanOrNot game, where users try to determine if they are interacting with a human or an AI.

This model was trained significantly faster (estimated 2x or more) and with lower memory usage thanks to Unsloth, leveraging its optimized kernels and integration with Huggingface's TRL library.

Training Details

Dataset

• Source: humanornot.ai
• Size: 1,000 curated chat conversations.
• Description: The dataset consists of dialogues where one participant is trying to guess whether the other is a human or an AI within a time limit. This often leads to unique questioning strategies, conversational styles, and potentially adversarial interactions.

Training Procedure

• Frameworks: Unsloth, Hugging Face TRL (Transformers Reinforcement Learning Library - likely used for Supervised Fine-tuning (SFT) in this context).
• Configuration:
  • Max Sequence Length: 2048 tokens
  • Epochs: 3
  • Total Training Steps: 60 (Note: This is a very small number of steps, a brief fine-tuning process.)
  • Quantization: Utilized the pre-quantized 4-bit base model (bnb-4bit).
• Hardware: Trained on 1x NVIDIA T4

Efficiency Gains

• Leveraging Unsloth's optimizations resulted in significantly faster training iterations and reduced GPU memory requirements compared to standard fine-tuning approaches with full precision or naive quantization.

Intended Use

• Primary Use: Research, experimentation, and qualitative exploration of fine-tuning LLMs on specific, niche conversational datasets like HumanOrNot.
• Potential Applications:
  • Simulating chat interactions resembling the HumanOrNot game.
  • Understanding how LLMs adapt to specific conversational constraints and objectives.
  • As a starting point for further fine-tuning on similar conversational tasks.

Note: Due to the limited size of the fine-tuning dataset (1k examples) and the very short training duration (60 steps), this model's capabilities might be narrowly focused on the style of the training data and may not generalize well to broader conversational tasks. It is not recommended for production use without further evaluation and potentially more extensive training.