H1dar1-Flux-LoKr

This is a LyCORIS adapter derived from black-forest-labs/FLUX.1-dev.

No validation prompt was used during training.

None

Validation settings

CFG: 3.0
CFG Rescale: 0.0
Steps: 20
Sampler: FlowMatchEulerDiscreteScheduler
Seed: 42
Resolution: 1632x688
Skip-layer guidance:

Note: The validation settings are not necessarily the same as the training settings.

You can find some example images in the following gallery:

Prompt
unconditional (blank prompt)

Negative Prompt
blurry, cropped, ugly

Prompt
Hidari Stop-Motion Animation: In a dimly lit, warmly lit interior with a textured golden-yellow wall behind, an aged figure resembling the Boss figure is depicted from the chest up in a medium shot. The figure exhibits the characteristic carved wood-like appearance and segmented construction. It wears dark robes with a detailed, shimmering gold patterned fabric and a thick, fluffy white ruff around its neck and framing its face. The figure's segmented hands, showing visible joints and a similar carved texture, are raised forward with palms open, positioned on either side of its head in a gesture that suggests surprise or pleading. Its face shows large, wide eyes with small dark pupils and a carved-open mouth. Blurred, ornate objects, possibly part of the setting, are visible in the foreground left and in the background. The lighting is focused, creating highlights on the face, hands, ruff, and robe against the warm, slightly blurred background.

Negative Prompt
blurry, cropped, ugly

Prompt
Hidari Stop-Motion Animation: In a close-up frame against a warm, diffuse yellow-orange background with dramatic lighting, the aged Boss Figure is prominently depicted. His face has a distinctive carved appearance with stark dark markings across his forehead and around his wide eyes. He has a prominent white beard and a ruff of white hair around his head, both appearing somewhat wispy. The figure holds a long, ornate pipe in his segmented, gray-toned hand, bringing it to his mouth which is slightly open, also appearing carved. A plume of white smoke is visible rising from the bowl of the pipe to the left. His dark, textured robe is visible at the bottom right of the frame. The lighting creates strong highlights on the figure's face, pipe, and hand, emphasizing the textures and forms characteristic of the stop-motion style.

Negative Prompt
blurry, cropped, ugly

Prompt
Hidari Stop-Motion Animation: In a high-contrast, dimly lit scene, the Boss figure, depicted with a carved wood-like face featuring sharp angles and visible grain, occupies the center foreground in a medium shot. He wears small round spectacles and dark, layered robes adorned with a distinctive white circular emblem containing a triangular arrangement of smaller circles on the left shoulder. A tuft of white textured material represents hair atop his head, from which a wrapped rope-like cord extends upwards out of frame, suggesting a marionette-like structure. The figure faces slightly right but gazes forward. Behind him and slightly to the right, a small, stylized animal figure resembling a cat, with patches of black and white fur and a visibly carved or molded form, sits perched upon a dark wooden surface. The background is largely dark, with subtle, warm illuminated rectangular areas suggesting distant light sources on the left and right.

Negative Prompt
blurry, cropped, ugly

Prompt
Hidari Stop-Motion Animation: In a high-contrast, dimly lit setting against a stark black background, two figures are posed in a dynamic combat sequence on a surface of weathered wooden planks. On the left, a figure resembling the protagonist, dressed in dark, layered, ragged clothing and featuring a distinctive topknot with a white tuft, leans forward aggressively, holding a wooden club-like weapon in its right hand, poised to strike. Opposite on the right, a red-robed figure wearing a pale, carved canine-skull mask is captured mid-lunge, wielding a curved sword overhead with both hands. Both figures clearly display visible segmentation in their limbs and body structure, contributing to a carved or constructed appearance. Dramatic, focused lighting illuminates the figures from the right, creating strong highlights and deep shadows that emphasize their forms and weapons. The frame is a medium-wide shot positioned at approximately eye level, capturing the full bodies of both combatants in their static, action-packed poses.

Negative Prompt
blurry, cropped, ugly

Prompt
Hidari Stop-Motion Animation: In a high-contrast, dimly lit setting against a black background, the protagonist figure is depicted in a dynamic pose, recoiling slightly. He wears dark, ragged layered clothing with lighter, segmented underlayers visible on his torso and circular symbols on his shoulder pieces. His face appears carved with distinct lines, and his hair is styled in a tall topknot with a prominent white tuft. His distinctive segmented mechanical left arm, exhibiting a wood-like texture with intricate joints, is raised high above his head, ending in a clenched fist. His normal right arm is bent forward, hand open and fingers splayed, also showing segmented joints. A glimpse of another figure's segmented hand is visible on the far left edge. The figure is strongly lit from the front and above by a focused spotlight, creating deep shadows and sharp highlights. The camera frames the figure from roughly the waist up in a medium shot.

Negative Prompt
blurry, cropped, ugly

Prompt
Hidari Stop-Motion Animation: In a dimly lit, high-contrast scene shot from a high angle looking downwards, the protagonist figure is depicted standing on a textured wooden plank floor. The figure is mostly covered in dark, layered, ragged fabric clothing with a visible circular symbol containing a triangular arrangement of crossed lines on the chest area. His prominent, segmented left arm, appearing to have a carved wood-like texture, is raised high, holding a large mechanical saw weapon aloft. The saw arm ends in a long blade with a visible metal chain and sharp teeth. Visible segmented parts of the figure's legs or feet are seen below the hem of the ragged clothing, also showing a wood-like texture. A strong spotlight illuminates the figure and the wooden floor below, creating stark highlights and deep shadows, with the background receding into blackness. The camera framing is a medium shot focused on the upper body and weapon.

Negative Prompt
blurry, cropped, ugly

The text encoder was not trained. You may reuse the base model text encoder for inference.

Training settings

Training epochs: 1
Training steps: 9500
Learning rate: 0.0004
- Learning rate schedule: polynomial
- Warmup steps: 100
Max grad value: 0.1
Effective batch size: 3
- Micro-batch size: 3
- Gradient accumulation steps: 1
- Number of GPUs: 1
Gradient checkpointing: True
Prediction type: flow_matching (extra parameters=['shift=3', 'flux_guidance_mode=constant', 'flux_guidance_value=1.0'])
Optimizer: adamw_bf16
Trainable parameter precision: Pure BF16
Base model precision: int8-quanto
Caption dropout probability: 0.05%
SageAttention: Enabled inference

LyCORIS Config:

{
    "algo": "lokr",
    "multiplier": 1.0,
    "linear_dim": 10000,
    "linear_alpha": 1,
    "factor": 16,
    "apply_preset": {
        "target_module": [
            "Attention",
            "FeedForward"
        ],
        "module_algo_map": {
            "Attention": {
                "factor": 16
            },
            "FeedForward": {
                "factor": 8
            }
        }
    }
}

Datasets

hidari_stills-256

Repeats: 7
Total number of images: 865
Total number of aspect buckets: 1
Resolution: 0.065536 megapixels
Cropped: False
Crop style: None
Crop aspect: None
Used for regularisation data: No

hidari_stills-crop-256

Repeats: 7
Total number of images: 865
Total number of aspect buckets: 1
Resolution: 0.065536 megapixels
Cropped: True
Crop style: center
Crop aspect: square
Used for regularisation data: No

hidari_stills-512

Repeats: 3
Total number of images: 865
Total number of aspect buckets: 1
Resolution: 0.262144 megapixels
Cropped: False
Crop style: None
Crop aspect: None
Used for regularisation data: No

hidari_stills-crop-512

Repeats: 3
Total number of images: 865
Total number of aspect buckets: 1
Resolution: 0.262144 megapixels
Cropped: True
Crop style: center
Crop aspect: square
Used for regularisation data: No

hidari_stills-768

Repeats: 1
Total number of images: 865
Total number of aspect buckets: 1
Resolution: 0.589824 megapixels
Cropped: False
Crop style: None
Crop aspect: None
Used for regularisation data: No

hidari_stills-crop-768

Repeats: 1
Total number of images: 865
Total number of aspect buckets: 1
Resolution: 0.589824 megapixels
Cropped: True
Crop style: center
Crop aspect: square
Used for regularisation data: No

hidari_stills-1024

Repeats: 0
Total number of images: 865
Total number of aspect buckets: 1
Resolution: 1.048576 megapixels
Cropped: False
Crop style: None
Crop aspect: None
Used for regularisation data: No

hidari_stills-crop-1024

Repeats: 0
Total number of images: 865
Total number of aspect buckets: 1
Resolution: 1.048576 megapixels
Cropped: True
Crop style: center
Crop aspect: square
Used for regularisation data: No

Inference

import torch
from diffusers import DiffusionPipeline
from lycoris import create_lycoris_from_weights


def download_adapter(repo_id: str):
    import os
    from huggingface_hub import hf_hub_download
    adapter_filename = "pytorch_lora_weights.safetensors"
    cache_dir = os.environ.get('HF_PATH', os.path.expanduser('~/.cache/huggingface/hub/models'))
    cleaned_adapter_path = repo_id.replace("/", "_").replace("\\", "_").replace(":", "_")
    path_to_adapter = os.path.join(cache_dir, cleaned_adapter_path)
    path_to_adapter_file = os.path.join(path_to_adapter, adapter_filename)
    os.makedirs(path_to_adapter, exist_ok=True)
    hf_hub_download(
        repo_id=repo_id, filename=adapter_filename, local_dir=path_to_adapter
    )

    return path_to_adapter_file
    
model_id = 'black-forest-labs/FLUX.1-dev'
adapter_repo_id = 'davidrd123/H1dar1-Flux-LoKr'
adapter_filename = 'pytorch_lora_weights.safetensors'
adapter_file_path = download_adapter(repo_id=adapter_repo_id)
pipeline = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16) # loading directly in bf16
lora_scale = 1.0
wrapper, _ = create_lycoris_from_weights(lora_scale, adapter_file_path, pipeline.transformer)
wrapper.merge_to()

prompt = "An astronaut is riding a horse through the jungles of Thailand."


## Optional: quantise the model to save on vram.
## Note: The model was quantised during training, and so it is recommended to do the same during inference time.
from optimum.quanto import quantize, freeze, qint8
quantize(pipeline.transformer, weights=qint8)
freeze(pipeline.transformer)
    
pipeline.to('cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu') # the pipeline is already in its target precision level
model_output = pipeline(
    prompt=prompt,
    num_inference_steps=20,
    generator=torch.Generator(device='cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu').manual_seed(42),
    width=1632,
    height=688,
    guidance_scale=3.0,
).images[0]

model_output.save("output.png", format="PNG")

davidrd123
/

H1dar1-Flux-LoKr