How to Deploy GPT-Load with Docker: Step-by-Step Tutorial

In this tutorial, you will deploy GPT-Load — the self-hosted AI gateway — with Docker, configure your first key group, and route traffic to OpenAI. By the end, you will have a production-ready setup with failover and usage tracking, all behind one endpoint.

Step 1: Run the Gateway with Docker

GPT-Load publishes an official image on Docker Hub with over 25K pulls. Start it with a single command:

docker run -d --name gpt-load -p 8080:8080 \
  -v ./data/gpt-load:/data \
  tbphp/gpt-load:latest

For a more structured setup, use docker-compose with a persistent volume so your keys and logs survive restarts:

services:
  gpt-load:
    image: tbphp/gpt-load:latest
    restart: unless-stopped
    ports:
      - 8080:8080
    volumes:
      - ./data/gpt-load:/data

Step 2: Create Your First Key Group

Open the management dashboard at http://localhost:8080 and log in with the admin credentials from your configuration. Go to Groups and create a group, for example "production". Inside the group, add your provider keys — this is where the multi-credential magic lives: you can mix API keys from multiple providers (OpenAI, Gemini, Claude) in one group, and GPT-Load will balance and fail over between them.

Step 3: Issue Access Keys

Under Access Keys, generate a gateway key bound to your group. This is the key your applications will use. Because the proxy is transparent, you keep calling the same OpenAI-compatible API format — just point your base URL at http://your-server:8080/v1 and swap the key. No code changes required.

Step 4: Verify Failover and Usage

Test failover by adding a second key to the group and temporarily revoking the first — requests should continue seamlessly. Finally, check the Usage & Cost dashboard to see per-key request logs and spending. This is your single source of truth for AI costs across every team and service.

Troubleshooting Tips

• Port already in use? Change the left side of 8080:8080 to any free port.
• Keys not rotating? Verify keys are placed in the correct group and the group is active.
• Configuration changes: GPT-Load hot-reloads settings, so no restart needed after edits.