TabbyAPI
Lightweight OpenAI-compatible ExLlamaV2 API server
★ 1,257 GitHub
AGPL-3.0 llminferenceexllamaapi-serveropenai-compatibleself-hostedquantization LLM & Chat
Überblick
TabbyAPI is the official API backend server for ExLlamaV2 and ExLlamaV3, providing a lightweight, fast FastAPI-based application for generating text using large language models. It offers a fully OpenAI-compatible API with support for loading/unloading models, HuggingFace model downloading, tool calling, LoRA adapters, and more. TabbyAPI publishes a CUDA-ready Docker image via GitHub Container Registry, making deployment straightforward on GPU-equipped servers. It is designed for hobbyist and small-scale use, with easy setup via Docker or manual Python installation.
Anforderungen
Min vCPU
2
Min RAM
4096 MB
Min Disk
10 GB
Rec vCPU
8
Rec RAM
16384 MB
Rec Disk
20 GB
Empfohlener VPS
Hostinger · KVM 8
8 vCPU · 32256 MB · 400 GB
Affiliate-Hinweis
Docker Compose
# Generated by Run This Ai — docker-compose.yml
services:
tabbyapi:
image: ghcr.io/theroyallab/tabbyapi:latest
restart: unless-stopped
ports:
- 8080:8080
volumes:
- ./data/tabbyapi:/data
Verwandte Tools
NextChat
Cross-platform ChatGPT web UI with multi-model support for Ollama, Claude, Gemini, and more
Lobe Chat
Extensible, open-source ChatGPT alternative with plugins, knowledge base, and multi-LLM support
Open WebUI
User-friendly WebUI for LLMs (Ollama, OpenAI API)
text-generation-webui
Run local LLMs with a powerful web interface — text, vision, tool-calling, and OpenAI-compatible API
Streamlit
Build and share data apps in pure Python — fast
Gradio
Build and share delightful machine learning apps in Python