Evidently

Evidently is an open-source Python framework for evaluating, testing, and monitoring ML and LLM-powered systems throughout their lifecycle. It works with both tabular and text data, supporting evals for predictive and generative tasks from classification to RAG. With 100+ built-in metrics covering data drift detection, model quality, LLM judges, and text evaluations, Evidently provides a comprehensive observability solution. The framework offers Reports for one-off evaluations and Test Suites with pass/fail conditions for CI/CD pipelines. For production monitoring, Evidently provides a self-hosted Monitoring UI dashboard that visualizes metrics and test results over time. Key capabilities include LLM evaluation with descriptors like sentiment analysis and text length checks, data drift detection using statistical methods, model performance tracking, and integration with existing ML pipelines through its Python interface and open architecture.