TensorZero

TensorZero is an open-source LLMOps platform that unifies an LLM gateway, observability, evaluation, optimization, and experimentation in a single system. It provides one endpoint for all major LLM providers, records every request and response pair, and lets you optimize prompts, model choices, and inference parameters based on real production traffic without custom glue code. TensorZero supports fine-tuning for supervised learning and reinforcement learning, A/B testing, and gradual rollout of new prompts and models. Built in Rust for high performance, it is licensed under Apache-2.0.