Colibri: Run Frontier 2.8T MoE Models on Hardware You Already Own

Frontier large language models keep getting bigger — 744B, 975B, even 2.8T parameters. The conventional wisdom says you need a rack of H100s to run them. Colibri disagrees: it's a pure-C inference engine that streams Mixture-of-Experts models from disk, letting you run frontier MoE models on the hardware you already own.

What is Colibri?

Colibri (colibrì — Italian for hummingbird) is a tiny inference engine built for immense models. Written in pure C with zero engine dependencies, it treats VRAM, RAM, and storage as a single multitier memory hierarchy. Instead of loading all 744B parameters into memory, it keeps experts on disk and streams them on demand — only the active experts are resident at any moment.

Models you can run today

The web dashboard shows a 744B model running at 4 tok/s with full expert residency on 6× RTX 5090 — with live token metrics and the VRAM/RAM/disk tier bar.

Why the memory hierarchy matters

Most inference engines fail when the model doesn't fit in VRAM. Colibri instead treats insufficient fast memory as a speed problem, never a correctness problem: it streams expert weights from NVMe storage on demand, caches hot experts in RAM, and keeps the working set in VRAM. A 744B GLM-5.2 model boots in ~32 seconds with only ~10 GB resident RAM. The engine also has a hard guarantee on semantics — the default policy never silently changes model precision or router behavior.

Three interfaces, one engine

coli chat gives you an interactive terminal chat. coli serve exposes an OpenAI-compatible API on port 5000, so it drops into existing tooling. coli web provides the live dashboard with the "Brain" page — a real-time cortex view of all 19,456 experts as they activate.

Colibri is both a practical engine and an open research platform for inference-side performance. If you've ever wished your GPU budget wasn't the bottleneck for using frontier models, this is the project to watch — and to run.