Run This Ai
EN DE

llama-cpp-python

Python bindings for llama.cpp with OpenAI-compatible server and multi-model support

★ 10,440 GitHub MIT llmpythonllama-cppinferenceopenai-compatibleself-hostedlocal-ai LLM & Chat

Overview

llama-cpp-python provides high-performance Python bindings for llama.cpp, the popular C/C++ LLM inference engine. With 10k+ GitHub stars, it offers a complete solution for running local LLMs: low-level ctypes access to the C API, a high-level Python API with LangChain/LlamaIndex compatibility, and an OpenAI-compatible web server with function calling, vision API, and multi-model support. Install via pip, run as a Docker container, or use as a local Copilot replacement — all fully self-hosted.

Requirements

Min vCPU
1
Min RAM
4096 MB
Min Disk
10 GB
Rec vCPU
4
Rec RAM
4096 MB
Rec Disk
20 GB

Recommended VPS

Hostinger · KVM 4

4 vCPU · 16384 MB · 200 GB

$11.99
View plan

Hostinger · KVM 4

4 vCPU · 16384 MB · 200 GB

$11.99
View plan

Hostinger · KVM 8

8 vCPU · 32256 MB · 400 GB

$20.00
View plan

Affiliate disclosure

Docker Compose

# Generated by Run This Ai — docker-compose.yml
services:
  llama-cpp-python:
    image: ghcr.io/abetlen/llama-cpp-python:latest
    restart: unless-stopped
    ports:
      - 8080:8080
    volumes:
      - ./data/llama-cpp-python:/data

Best VPS for llama-cpp-python →

Related tools

NextChat

Cross-platform ChatGPT web UI with multi-model support for Ollama, Claude, Gemini, and more

Lobe Chat

Extensible, open-source ChatGPT alternative with plugins, knowledge base, and multi-LLM support

Open WebUI

User-friendly WebUI for LLMs (Ollama, OpenAI API)

text-generation-webui

Run local LLMs with a powerful web interface — text, vision, tool-calling, and OpenAI-compatible API

Streamlit

Build and share data apps in pure Python — fast

Gradio

Build and share delightful machine learning apps in Python

Guides & articles

Getting Started with llama-cpp-python OpenAI-Compatible Server

Step-by-step tutorial to install, configure, and run llama-cpp-python's OpenAI-compatible LLM server with your own models.

Jun 27, 2026

How to Run LLMs Locally with llama-cpp-python - A Complete Guide

A complete guide to running LLMs locally with llama-cpp-python — OpenAI-compatible server, vision support, function calling, and multi-model serving.

Jun 27, 2026