Getting Started with LangServe: Docker Quick Start Guide

Running LangServe with Docker

LangServe is the easiest way to deploy LangChain runnables as REST APIs, and Docker is the simplest way to run it. This guide walks you through setting up LangServe with Docker for production-ready API deployments.

Prerequisites

• Docker and Docker Compose installed on your server
• A LangChain runnable or chain to deploy
• Basic familiarity with Python and FastAPI

Quick Start with Docker

The official LangServe Docker image is available on Docker Hub as langchain/langserve:latest. Here is how to get started:

# Pull the image
docker pull langchain/langserve:latest

# Run the container
docker run -d --name langserve -p 8080:8080 \
  -v $(pwd)/app:/app \
  langchain/langserve:latest

This starts LangServe on port 8080 with your application code mounted from the ./app directory.

Docker Compose Setup

For a more complete setup, use Docker Compose:

version: "3.8"
services:
  langserve:
    image: langchain/langserve:latest
    restart: unless-stopped
    ports:
      - "8080:8080"
    volumes:
      - ./app:/app
      - ./data:/data

Building Your LangServe App

Create a simple app/server.py file with your LangChain chain:

#!/usr/bin/env python3
from fastapi import FastAPI
from langserve import add_routes
from langchain.llms import OpenAI
from langchain.prompts import PromptTemplate
from langchain.chains import LLMChain

app = FastAPI(title="LangServe API")

prompt = PromptTemplate(
    input_variables=["topic"],
    template="Tell me an interesting fact about {topic}."
)
llm = OpenAI(temperature=0.7)
chain = LLMChain(llm=llm, prompt=prompt)

add_routes(app, chain, path="/facts")

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=8080)

Testing Your API

Once the container is running, test your endpoint:

curl -X POST http://localhost:8080/facts/invoke \
  -H "Content-Type: application/json" \
  -d '{"input": {"topic": "elephants"}}'

You should receive a JSON response with the generated fact. You can also open the Playground UI at http://localhost:8080/facts/playground/ in your browser to interact with the API visually.

Streaming Responses

LangServe supports streaming out of the box. Use the streaming endpoint for real-time responses:

curl -N http://localhost:8080/facts/stream \
  -H "Content-Type: application/json" \
  -d '{"input": {"topic": "space"}}'

Conclusion

Running LangServe with Docker gives you a portable, scalable way to deploy LangChain APIs. Whether you are building a simple chatbot or a complex RAG pipeline, the Docker setup lets you focus on your chain logic while LangServe handles the API layer.