Run This Ai
EN DE

Getting Started with LLaVA: Run Multimodal AI on Your Own Machine

Learn to run LLaVA locally: llama.cpp, official repo, or Gradio UI. Supports consumer GPUs with 4-bit quantization.

LLaVA Logo

Quick Start with LLaVA

LLaVA is one of the most accessible multimodal AI models to run locally. Whether you have a consumer GPU or a high-end workstation, you can have LLaVA answering questions about images within minutes. Here are three ways to get started.

Method 1: llama.cpp (Easiest)

The fastest path to running LLaVA is through llama.cpp, which supports 4-bit and 5-bit quantization for the 7B model on as little as 6GB VRAM.

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && make
wget https://huggingface.co/mys/ggml_llava-v1.5-7b/resolve/main/ggml-model-q4_k.gguf
wget https://huggingface.co/mys/ggml_llava-v1.5-7b/resolve/main/mmproj-model-f16.gguf
./server -m ggml-model-q4_k.gguf --mmproj mmproj-model-f16.gguf --host 0.0.0.0 --port 8080

Then visit http://localhost:8080 to upload images and chat with LLaVA.

LLaVA GitHub

Method 2: Official Repository (Full Control)

For the latest models and full flexibility, use the official codebase with Python 3.10+ and PyTorch.

git clone https://github.com/haotian-liu/LLaVA
cd LLaVA
conda create -n llava python=3.10 && conda activate llava
pip install -e .
python -m llava.serve.cli --model-path liuhaotian/llava-v1.5-7b --image-file photo.jpg

Method 3: Gradio Web UI

Launch a browser interface for image uploads and chat conversations.

python -m llava.serve.gradio_web_server --model-path liuhaotian/llava-v1.5-7b

Opens at http://localhost:7860 with an intuitive drag-and-drop interface.

Hardware Guide

7B (4-bit): 6GB VRAM, RTX 3060. 7B (FP16): 16GB, RTX 3090. 13B (4-bit): 10GB, RTX 3090. 34B (4-bit): 20GB, A100.

Fine-Tuning

LLaVA supports LoRA fine-tuning on consumer GPUs. Full scripts are provided for data preparation, training, and evaluation. Fine-tune the 7B model on custom datasets in under 24 hours on a single RTX 3090.

Conclusion

From the simplicity of llama.cpp to the flexibility of the official codebase, there is a LLaVA setup for every skill level and budget. The thriving community and regular model releases keep pushing the boundaries of open-source multimodal AI.