---
datasets:
- Eunju2834/img_captioning_oilcanvas_style
- Eunju2834/oil_impressionism_style
pipeline_tag: text-to-image
base_model: runwayml/stable-diffusion-v1-5
tags:
- LoRA
- Diffusion
- OilCanvas
- stable-diffusion
- text-to-image
license: creativeml-openrail-m
---

# 🎨 LoRA text2image fine-tuning - Oil Canvas Style

## 📄 Model Description 
This model is a fine-tuned version of the Stable Diffusion v1.5 model using Low-Rank Adaptation (LoRA) techniques to generate images in an oil canvas painting style, specifically focusing on the Impressionism genre. The model is designed to produce vibrant, brushstroke-rich images with a joyful and community-focused theme.

## 🔍 Model Details
- **Base Model:** Stable Diffusion v1.5 (`runwayml/stable-diffusion-v1-5`)
- **Fine-Tuning Method:** LoRA (Low-Rank Adaptation)
- **Training Data:** Custom oil canvas style dataset collected from Kaggle, focused on Impressionism artworks (`Eunju2834/img_captioning_oilcanvas_style`,`Eunju2834/oil_impressionism_style`)
- **Captioning Method:** BLIP-2 model used to generate image captions for the dataset
- **Training Configuration:**
  - Epochs: 20
  - Batch Size: 1
  - Learning Rate: 1e-4
  - Scheduler: Cosine
  - Seed: 2024

## 🚀 Usage
```python
from diffusers import StableDiffusionPipeline
import torch

model_path = 'Eunju2834/LoRA_oilcanvas_style'
pipe = StableDiffusionPipeline.from_pretrained('runwayml/stable-diffusion-v1-5', torch_dtype=torch.float16, use_auth_token=True)
pipe.unet.load_attn_procs(model_path)
pipe.to('cuda')

prompt = '''(Oil Painting: 1.1), (Impressionism: 1.2), (oil painting with brush strokes: 1.2),
Park stroll, joyful atmosphere, laughter-filled time, playful dogs, vibrant park scene,
cheerful interactions, happy pet owners, heartwarming moments, vibrant community vibes'''

neg_prompt = '''FastNegativeV2, (bad-artist:1.0), (worst quality, low quality:1.4),
(watermark), error, missing fingers, extra digit, cropped, normal quality, blurry'''

image = pipe(prompt, negative_prompt=neg_prompt, num_inference_steps=30, guidance_scale=7.5).images[0]
image.save('oil_impressionism_park_stroll.png')
```

## 🖼️ Example Results
The model generates images like the ones below, showcasing an oil painting style with vibrant colors and Impressionist influences.
<p style="display: flex; justify-content: center;">
  <img src="https://cdn-uploads.huggingface.co/production/uploads/64bad345f671da974eeb1ba3/SfjQLeVKie31DZSB23jrq.png" width="250px" />
  <img src="https://cdn-uploads.huggingface.co/production/uploads/64bad345f671da974eeb1ba3/sUTDFT20kGhrRUilSgVOe.png" width="250px" />
  <img src="https://cdn-uploads.huggingface.co/production/uploads/64bad345f671da974eeb1ba3/MhGz7DPm_Eb_NWoIxQx28.png" width="250px" />
</p>



## ⚠️ Limitations and Biases
- The model is optimized for oil canvas style images and may not generalize well to other artistic styles.
- Potential biases may exist due to the specific nature of the training dataset (e.g., Impressionism artworks).

## 📜 License
CreativeML Open RAIL-M