Getting Started with LLM-Ware: From Document to RAG Query in Minutes

Getting Started with LLM-Ware: From Document to RAG Query in Minutes

One of the best things about LLM-Ware is how quickly you can go from raw documents to a working RAG pipeline. In this tutorial, we'll walk through the essential steps: installing the framework, parsing a document, creating embeddings, and running your first query.

Prerequisites

LLM-Ware runs on Python 3.10+ on Linux or macOS (Windows support via WSL). You'll need at least 1 GB of RAM for basic document processing, and 4+ GB for running local models. A CUDA-capable GPU is optional but helpful for larger embedding models.

Installation

Installing LLM-Ware is straightforward with pip:

pip install llmware

That's it. The framework bundles its own document parsers, so there are no heavy system dependencies to install separately. For optional speed improvements with larger datasets, you can also install the FAISS backend:

pip install llmware[faiss]

Step 1: Parse a Document

Let's start by parsing a PDF. LLM-Ware's Library class handles document ingestion:

from llmware.library import Library

# Create a library to hold our documents
lib = Library().create_new_library("my_docs")

# Add files from a directory
lib.add_files("/path/to/documents",
              chunk_size=400,
              use_llmware_parser=True)

The parser extracts text, tables, and metadata from each document. The chunk_size parameter controls how the text is split into overlapping segments for retrieval.

Step 2: Generate Embeddings

Once documents are parsed and chunked, we need to create embeddings. LLM-Ware provides a simple API for this:

from llmware.embeddings import EmbeddingsHandler

emb = EmbeddingsHandler(library=lib)
emb.embed_custom(embedding_model_name="mini-lm-sbert",
                 vector_db="faiss")

This generates embeddings for all chunks using a lightweight Sentence-BERT model and stores them in a FAISS index. The entire process — parsing, chunking, and embedding a 100-page document — typically takes under a minute on a modern CPU.

Step 3: Query Your Documents

With embeddings ready, you can start asking questions:

from llmware.retrieval import Retrieval

retriever = Retrieval(library=lib)
results = retriever.text_query("What are the key features of our product?")

for result in results:
    print(f"Source: {result['file_source']}")
    print(f"Text: {result['text'][:300]}...")
    print(f"Score: {result['distance']:.4f}\n")

The retriever performs semantic search over your document chunks, returning the most relevant passages along with source metadata and similarity scores.

Step 4: Generate Answers with RAG

To close the loop, use a small local model to generate answers based on retrieved context:

from llmware.prompts import Prompt

prompter = Prompt()
prompter.load_model("phi-3-mini-128k-instruct",
                    temperature=0.3,
                    max_output=200)

context = "\n".join([r["text"] for r in results[:3]])
answer = prompter.inference(
    prompt="Based on the following context, answer the question.",
    context=context,
    query="What are the key features of our product?"
)
print(answer)

LLM-Ware integrates with Phi-3, Llama-3, Gemma, and other small models that run efficiently on consumer hardware. No API keys or cloud services needed.

Why This Matters for Enterprise

The entire pipeline above runs completely offline, on your own infrastructure. That means sensitive documents never leave your network, there are no per-token API costs, and you can scale horizontally by adding more worker nodes. LLM-Ware also tracks provenance — every answer is traceable back to its source document and chunk — which is essential for compliance and audit requirements.

Next Steps

Once you've got the basic pipeline running, explore LLM-Ware's more advanced features: custom embedding models, hybrid search (semantic + keyword), reranking pipelines, multi-library queries, and scheduled re-indexing. The official documentation covers all of these in depth.

Conclusion

LLM-Ware makes enterprise RAG genuinely accessible. With just a few lines of Python, you can go from raw PDFs to a fully functional semantic search and question-answering system — all running locally with small, efficient models. It's the kind of tool that every organization dealing with large document collections should have in their toolkit.