Getting Started with Kohya SS: Train Your First LoRA Model

Getting Started with Kohya SS: Train Your First LoRA Model

Ready to create your own custom Stable Diffusion model? This guide walks you through deploying Kohya SS with Docker and running your first LoRA training session. By the end, you'll have a working training environment and understand the core workflow.

Prerequisites

Before you begin, make sure you have:

• A GPU with at least 8 GB VRAM — 12 GB or more recommended for SDXL training
• Docker and Docker Compose installed on your system
• NVIDIA Container Toolkit (nvidia-docker2) for GPU passthrough
• 20+ GB of free disk space for base models and training data

Step 1: Deploy Kohya SS with Docker

The ashleykza/kohya image bundles CUDA, PyTorch, xformers, and all dependencies into a single container. Pull it and run:

# Pull the image
docker pull ashleykza/kohya:latest

# Run the container with GPU access
docker run -d \
  --name kohya-ss \
  --gpus all \
  -p 7860:7860 \
  -v ~/kohya_data:/workspace \
  ashleykza/kohya:latest

Once the container starts, open your browser and navigate to http://localhost:7860. You'll see the Kohya SS Gradio interface — a tabbed dashboard with all the training tools at your fingertips.

Step 2: Prepare Your Training Data

Kohya SS needs three things to train a model:

1. A base model. Download a Stable Diffusion checkpoint (SD 1.5, SDXL, etc.) and place it in your ~/kohya_data directory. Popular choices include sd_xl_base_1.0.safetensors or community fine-tunes.
2. Training images. Gather 10–50 high-quality images of your subject (character, style, object). Place them in a folder like ~/kohya_data/dataset/my_subject/. More variety = better generalization; fewer images = faster training but risk of overfitting.
3. Captions. Each image needs a text description. Use Kohya SS's built-in Utilities tab to auto-caption with BLIP or WD14 Tagger, or write them manually. Good captions dramatically improve LoRA quality — describe what you don't want the model to learn as well as what you do.

Step 3: Configure and Launch Training

Switch to the LoRA tab in the UI. Here's a battle-tested starting configuration:

• Base Model: Choose SDXL or SD 1.5 depending on your checkpoint
• Network Rank (Dimension): 32 for style LoRAs, 64–128 for character LoRAs
• Network Alpha: Half the rank (16 for rank 32)
• Learning Rate: 0.0001 (1e-4) — the safe default
• Epochs: 10–20 for 20-50 images
• Batch Size: 1 (if you have limited VRAM)
• Optimizer: AdamW8bit
• Mixed Precision: fp16 or bf16

Click "Start Training" and watch the console output. Kohya SS will generate sample images at configurable intervals so you can visually track how your model improves epoch by epoch. Training typically takes 15–60 minutes depending on your GPU and dataset size.

Step 4: Use Your Trained LoRA

Once training completes, find your .safetensors LoRA file in the output directory. Load it into Automatic1111, ComfyUI, or any inference tool that supports LoRA weights. A typical prompt for a character LoRA might look like:

<lora:my_character:0.8> 
portrait of my_character, studio lighting, detailed face, 8k

Troubleshooting Common Issues

• CUDA out of memory: Reduce batch size to 1, lower network rank, or use a smaller base model (SD 1.5 instead of SDXL).
• Overfitting (same face every time): Reduce epochs, increase dataset variety, or lower network rank.
• Underfitting (LoRA has no effect): Increase epochs, raise learning rate slightly, or increase network rank.
• Docker can't see GPU: Verify nvidia-smi works on the host and that --gpus all flag is passed. Install nvidia-container-toolkit if missing.

Kohya SS is deep water — there's always more to learn about advanced techniques like masked loss, multi-GPU training, and learning rate scheduling. But this workflow covers the 80% use case and will get you from zero to a working custom model in under an hour.