How to Install ODS: Set Up a Local AI Server with LLM, Voice & RAG

Getting a local AI server running used to be a weekend project. With ODS, it's a coffee-break project: download the installer, pick your platform, and you get LLM inference, voice, agents, and RAG in one go. Here's the full walkthrough.

Step 1: Pick Your Variant

ODS publishes multiple docker-compose files, so choose the one that matches your hardware:

• CPU-only — perfect for testing on any laptop or desktop
• NVIDIA — for CUDA-accelerated inference on RTX cards
• AMD — for ROCm-based GPUs
• Apple Silicon — optimized for M1/M2/M3 Macs
• Cloud — pre-tuned for VPS and cloud instances

Step 2: Install & Start

Download the ODS installer from the official site (a Tauri desktop app for Windows, macOS, and Linux), launch it, and follow the prompts. The installer detects your platform, pulls the right compose file, and starts the stack. If you prefer the terminal route, the Docker Compose files are on the GitHub repo — clone, pick your variant, and run docker compose up -d.

Step 3: First Boot

Once the services are up, open the web UI at http://localhost:8080. From there you can:

• Pick an LLM model and start chatting immediately
• Enable STT/TTS for voice conversations
• Connect your documents to RAG for Q&A over your files
• Open n8n to build automation workflows, or ComfyUI for image generation

What You Need

ODS is the fastest path we've found from "I want a local AI server" to "it's running." The complete requirements and compose setup are on the tool page below.