Evidently

Evidently

Evidently is an open-source Python library for evaluating, testing, and monitoring ML models and LLM-powered applications. It provides 100+ built-in metrics covering data quality, data drift, model performance, and LLM output quality.

Key Features

• LLM evaluation: Assess text quality, hallucination, toxicity, semantic similarity, and custom criteria
• Data drift detection: Statistical tests (KS, PSI, Wasserstein) to detect distribution shifts in features
• Column-level reports: Generate interactive HTML reports for any dataset or prediction batch
• Test suites: Codify quality expectations as pass/fail tests for CI/CD integration
• Monitoring platform: Evidently Cloud or self-hosted for continuous production monitoring
• Integrations: Works with MLflow, Airflow, Prefect, Dagster, and any Python-based pipeline

Quick Start

pip install evidently

from evidently import Dataset, DataDefinition
from evidently.presets import DataDriftPreset
from evidently import Report

report = Report(metrics=[DataDriftPreset()])
report.run(reference_data=reference_df, current_data=current_df)
report.save_html("drift_report.html")

Add to ai-supply

npx ai-supply add evidently-ml-monitoring

Curated mirror of the open-source Evidently (Apache-2.0). Get it from the source.