SillyTavern on Ubuntu 24.04 + Ollama

What is SillyTavern?

SillyTavern is a powerful UI for LLM roleplay and chat — customizable characters, group chats, plugins, and lorebooks. Paired with Ollama, it's a fully private self-hosted AI chat experience.

Prerequisites

• Ubuntu 24.04 LTS, 2 CPU / 2 GB RAM (4+ GB recommended)
• Docker Engine + docker-compose plugin
• Ollama already running (systemctl status ollama)
• Port 8000 free on the host

1 — Install Docker

sudo apt-get update && sudo apt-get install -y ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc && sudo chmod a+r /etc/apt/keyrings/docker.asc
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu noble stable" | sudo tee /etc/apt/sources.list.d/docker.list
sudo apt-get update && sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin
sudo usermod -aG docker $USER

2 — Project Setup

mkdir -p ~/sillytavern/{data,plugins}
cd ~/sillytavern

Create docker-compose.yml:

version: "3.8"
services:
  sillytavern:
    image: ghcr.io/sillytavern/sillytavern:latest
    container_name: sillytavern
    restart: unless-stopped
    ports:
      - "8000:8000"
    volumes:
      - ./data:/home/node/app/data
      - ./plugins:/home/node/app/plugins
    environment:
      - SILENT=true
    extra_hosts:
      - "host.docker.internal:host-gateway"

3 — Start

docker compose up -d

Open http://localhost:8000. For Nginx:

sudo apt-get install -y nginx
cat <<'EOF' | sudo tee /etc/nginx/sites-available/sillytavern
server {
    listen 80;
    server_name sillytavern.example.com;
    location / {
        proxy_pass http://127.0.0.1:8000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
    }
}
EOF
sudo ln -s /etc/nginx/sites-available/sillytavern /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx

Env Variables

• SILENT=true — suppress startup banner (default: false)
• DISABLE_PLUGINS=true — disable all plugins for troubleshooting (default: false)
• CONFIG_PATH — override config file location (default: data/config.yaml)

4 — Connect to Ollama

1. Click the plug icon (top-left).
2. Set API Type to Ollama.
3. Set API URL to http://host.docker.internal:11434 (or LAN IP).
4. Click Connect.

5 — Model Config

ollama pull llama3.2:3b
ollama pull mistral:7b

Pick a model, set Context Size (4096), adjust Temperature (0.7–1.2 for creative writing). Switch models mid-session.

6 — RAG with Lorebooks

Click the book icon (right sidebar). Import or create entries with keywords and content. Attach to characters or globally. Matched entries inject context automatically.

Backup & Update

# Backup (characters, chats, lorebooks)
tar -czf st-$(date +%Y%m%d).tar.gz ~/sillytavern/data
# Update
cd ~/sillytavern && docker compose pull && docker compose up -d && docker image prune -f

Data persists across updates.

Troubleshooting

Conn refused

• Check systemctl status ollama.
• Set OLLAMA_HOST=0.0.0.0:11434 in ollama.service, restart.
• Test: docker exec sillytavern curl -s http://host.docker.internal:11434/api/tags.
• Open firewall: sudo ufw allow 11434.

Slow on large contexts

• Reduce Context Size to 4096.
• Trim lorebook entries.
• Use a smaller model (3B–7B).
• Add a GPU for larger models.

Plugin issues

• Check Plugin Console for errors.
• Set DISABLE_PLUGINS=true to isolate.
• Remove plugins from plugins/, restart.

Character data lost

• Verify ./data volume mount is correct.
• Restore: tar -xzf st-*.tar.gz -C ~/sillytavern.
• Enable auto-save in Settings > Chat (1–5 min).