How to Deploy Faster-Whisper with Docker: Complete Transcription Tutorial

How to Deploy Faster-Whisper with Docker: Complete Transcription Tutorial

This tutorial walks you through deploying Faster-Whisper with Docker — from pulling the image to setting up a transcription API and optimizing for production. No prior ML experience needed.

Prerequisites

• Docker installed on your system
• NVIDIA GPU with 2GB+ VRAM (for GPU mode) or CPU is fine (slower but works)
• NVIDIA Container Toolkit (for GPU acceleration in Docker)
• Audio files you want to transcribe (MP3, WAV, M4A, FLAC)

Step 1: Pull the Docker Image

docker pull linuxserver/faster-whisper:latest

This image from LinuxServer.io comes with Faster-Whisper, CUDA, cuDNN, and all dependencies pre-installed. It's about 4GB compressed and includes model caching for faster first-run.

Step 2: Create docker-compose.yml

version: '3.8'
services:
  faster-whisper:
    image: linuxserver/faster-whisper:latest
    restart: unless-stopped
    ports:
      - "9000:9000"
    volumes:
      - ./models:/models
      - ./audio:/audio
      - ./output:/output
    environment:
      - WHISPER_MODEL=large-v3
      - WHISPER_BEAM_SIZE=5
      - WHISPER_COMPUTE_TYPE=int8_float16
      - WHISPER_DEVICE=cuda
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

What each environment variable does:

• WHISPER_MODEL — which Whisper model to use (tiny through large-v3)
• WHISPER_BEAM_SIZE — search beam width (5=good quality, 1=fastest)
• WHISPER_COMPUTE_TYPE — precision: float16 (fast), int8_float16 (lowest VRAM), float32 (most accurate)
• WHISPER_DEVICE — cuda for GPU, cpu for CPU-only

Step 3: Start and Transcribe

mkdir -p models audio output
cp meeting.wav ./audio/
docker compose up -d

# Transcribe via API (default FastAPI)
curl -X POST http://localhost:9000/asr \
  -F "audio_file=@./audio/meeting.wav" \
  -F "model=large-v3" | python3 -m json.tool

Step 4: Using the Python API Script

docker exec faster-whisper python3 << 'EOF'
from faster_whisper import WhisperModel

model = WhisperModel("large-v3", device="cuda", compute_type="int8_float16")
segments, info = model.transcribe("/audio/meeting.wav", beam_size=5)

for seg in segments:
    print(f"[{seg.start:.2f}s -> {seg.end:.2f}s] {seg.text}")
EOF

Step 5: VAD-Enhanced Transcription

docker exec faster-whisper python3 << 'EOF'
from faster_whisper import WhisperModel

model = WhisperModel("large-v3", device="cuda", compute_type="float16")
segments, _ = model.transcribe("/audio/meeting.wav", beam_size=5,
    vad_filter=True,  # Enable VAD filtering
    vad_parameters=dict(min_silence_duration_ms=500))

for seg in segments:
    print(f"[{seg.start:.2f}s -> {seg.end:.2f}s] {seg.text}")
EOF

VAD filtering significantly improves accuracy by removing silence hallucinations.

Optimization Guide

Troubleshooting

Problem: "CUDA out of memory"

Solution: Switch to int8_float16 compute type or use a smaller model (medium instead of large-v3). Also try setting WHISPER_BEAM_SIZE=1 for lower memory usage.

Problem: Container won't start on CPU

Solution: Set WHISPER_DEVICE=cpu and WHISPER_COMPUTE_TYPE=int8. Remove the deploy.resources section from docker-compose.yml.

Problem: No GPU detected inside container

Solution: Run docker exec faster-whisper nvidia-smi to verify GPU access. If not found, install NVIDIA Container Toolkit: sudo apt install nvidia-container-toolkit && sudo systemctl restart docker.

Problem: Very slow transcription

Solution: On GPU, use float16 compute type. On CPU, use int8 and set cpu_threads to your core count. Also try a smaller model.

Building an API Server

For production, create a simple Flask/FastAPI wrapper:

# save as api.py
from faster_whisper import WhisperModel
from flask import Flask, request, jsonify
import tempfile, os

app = Flask(__name__)
model = WhisperModel("large-v3", device="cuda", compute_type="float16")

@app.route("/transcribe", methods=["POST"])
def transcribe():
    f = request.files["audio"]
    tmp = tempfile.NamedTemporaryFile(delete=False, suffix=".wav")
    f.save(tmp.name)

    segments, info = model.transcribe(tmp.name, beam_size=5, vad_filter=True)
    results = [{"start": s.start, "end": s.end, "text": s.text} for s in segments]

    os.unlink(tmp.name)
    return jsonify({"language": info.language, "segments": results, "duration": info.duration})

app.run(host="0.0.0.0", port=9000)

Conclusion

With Docker, Faster-Whisper takes about 10 minutes to set up and delivers production-grade transcription. The key is choosing the right model and compute type for your hardware — the int8_float16 mode on large-v3 gives an excellent balance of accuracy and VRAM efficiency that no other Whisper implementation matches.