vLLM Ascend
High-throughput LLM serving on Huawei Ascend NPUs. Community-maintained hardware plugin for vLLM.
Overview
Requirements
Docker Compose
# Generated by Run This Ai — docker-compose.yml
services:
vllm-ascend:
image: ascendai/vllm-ascend:latest
restart: unless-stopped
ports:
- 8080:8080
volumes:
- ./data/vllm-ascend:/data
Related tools
NextChat
Cross-platform ChatGPT web UI with multi-model support for Ollama, Claude, Gemini, and more
Lobe Chat
Extensible, open-source ChatGPT alternative with plugins, knowledge base, and multi-LLM support
Open WebUI
User-friendly WebUI for LLMs (Ollama, OpenAI API)
text-generation-webui
Run local LLMs with a powerful web interface — text, vision, tool-calling, and OpenAI-compatible API
Streamlit
Build and share data apps in pure Python — fast
Gradio
Build and share delightful machine learning apps in Python
Guides & articles
How to Deploy vLLM Ascend with Docker: Step-by-Step Tutorial
Deploy vLLM on Huawei Ascend NPUs in minutes with the official ascendai/vllm-ascend Docker image. Pull, run, and query an OpenAI-compatible LLM endpoint.
vLLM Ascend: Run High-Throughput LLM Inference on Huawei Ascend NPUs
vLLM Ascend brings the vLLM inference engine to Huawei Ascend NPUs. Explore features, hardware support, and why Ascend is a serious GPU alternative for self-hosted LLM serving.