Run This Ai
EN DE

Xinference

Run open-source LLMs, embeddings, and multimodal models with one line of code

★ 9,389 GitHub Apache-2.0 llm-servinginferenceembeddingsmultimodalself-hostedopen-source LLM & Chat

Overview

Xinference (Xorbits Inference) is a powerful open-source platform for deploying and running large language models, embedding models, and multimodal models on your own infrastructure. With 9k+ GitHub stars, Xinference provides a unified API for serving models from Hugging Face, ModelScope, and custom checkpoints. Key features include distributed serving across multiple machines, a built-in web UI for model management, full OpenAI-compatible API endpoints, GPU and CPU support, and built-in quantization. Simply change one line of code to swap between GPT-4 and any open-source model, making it ideal for development, testing, and production deployments.

Requirements

Min vCPU
1
Min RAM
4096 MB
Min Disk
10 GB
Rec vCPU
2
Rec RAM
8192 MB
Rec Disk
20 GB

Recommended VPS

Hostinger · KVM 2

2 vCPU · 8192 MB · 100 GB

$6.99
View plan

Hostinger · KVM 2

2 vCPU · 8192 MB · 100 GB

$6.99
View plan

Hostinger · KVM 4

4 vCPU · 16384 MB · 200 GB

$11.99
View plan

Affiliate disclosure

Docker Compose

# Generated by Run This Ai — docker-compose.yml
services:
  xinference:
    image: xprobe/xinference:latest
    restart: unless-stopped
    ports:
      - 8080:8080
    volumes:
      - ./data/xinference:/data

Best VPS for Xinference →

Related tools

NextChat

Cross-platform ChatGPT web UI with multi-model support for Ollama, Claude, Gemini, and more

Lobe Chat

Extensible, open-source ChatGPT alternative with plugins, knowledge base, and multi-LLM support

Open WebUI

User-friendly WebUI for LLMs (Ollama, OpenAI API)

text-generation-webui

Run local LLMs with a powerful web interface — text, vision, tool-calling, and OpenAI-compatible API

Streamlit

Build and share data apps in pure Python — fast

Gradio

Build and share delightful machine learning apps in Python

Guides & articles

Xinference vs TGI: Choosing the Right LLM Server for Self-Hosting

Detailed comparison of Xinference vs Text Generation Inference (TGI): model support, performance, features, and recommendations for choosing the right LLM server.

Jun 27, 2026

Getting Started with Xinference: One CLI to Rule All LLMs

Complete guide to getting started with Xinference: deploy LLMs, embeddings, and multimodal models with one CLI command. Docker setup, pip install, and LangChain integration.

Jun 27, 2026