Run This Ai
EN DE

Text Generation Inference

Hugging Face's high-performance LLM serving with Rust/Python for production

★ 10,862 GitHub Apache-2.0 llm-servinginferencehuggingfacetext-generationself-hostedtransformer LLM & Chat

Überblick

Text Generation Inference (TGI) is Hugging Face's production-grade serving solution for Large Language Models. Built with Rust and Python, TGI delivers blazing-fast inference with features like tensor parallelism, continuous batching, streaming tokens, and quantization support (bitsandbytes, GPTQ, AWQ). It supports all major open-source LLMs and provides an OpenAI-compatible API endpoint. With 10k+ GitHub stars, TGI is the go-to deployment solution for enterprises and developers running LLMs at scale in production environments. It integrates seamlessly with the Hugging Face ecosystem, allowing one-command deployment of thousands of models from the Hub.

Anforderungen

Min vCPU
2
Min RAM
4096 MB
Min Disk
10 GB
Rec vCPU
4
Rec RAM
16384 MB
Rec Disk
20 GB

Empfohlener VPS

Hostinger · KVM 4

4 vCPU · 16384 MB · 200 GB

$11.99
Zum Anbieter

Hostinger · KVM 4

4 vCPU · 16384 MB · 200 GB

$11.99
Zum Anbieter

Hostinger · KVM 8

8 vCPU · 32256 MB · 400 GB

$20.00
Zum Anbieter

Affiliate-Hinweis

Docker Compose

# Generated by Run This Ai — docker-compose.yml
services:
  text-generation-inference:
    image: ghcr.io/huggingface/text-generation-inference:latest
    restart: unless-stopped
    ports:
      - 8080:8080
    volumes:
      - ./data/text-generation-inference:/data

Bester VPS für Text Generation Inference →

Verwandte Tools

NextChat

Cross-platform ChatGPT web UI with multi-model support for Ollama, Claude, Gemini, and more

Lobe Chat

Extensible, open-source ChatGPT alternative with plugins, knowledge base, and multi-LLM support

Open WebUI

User-friendly WebUI for LLMs (Ollama, OpenAI API)

text-generation-webui

Run local LLMs with a powerful web interface — text, vision, tool-calling, and OpenAI-compatible API

Streamlit

Build and share data apps in pure Python — fast

Gradio

Build and share delightful machine learning apps in Python

Anleitungen & Artikel

Xinference vs TGI: Choosing the Right LLM Server for Self-Hosting

Detailed comparison of Xinference vs Text Generation Inference (TGI): model support, performance, features, and recommendations for choosing the right LLM server.

Jun 27, 2026

Text Generation Inference Review: Performance, Features, and Verdict

In-depth review of Hugging Face Text Generation Inference (TGI): performance benchmarks, quantization support, strengths, weaknesses, and verdict for self-hosted LLM serving.

Jun 27, 2026

How to Deploy Text Generation Inference (TGI) with Docker

Complete guide to deploying Hugging Face Text Generation Inference (TGI) with Docker. Covers prerequisites, Docker setup, API usage, and production configuration.

Jun 27, 2026