RAGAS

RAGAS

RAGAS (Retrieval Augmented Generation Assessment) is an open-source framework for evaluating RAG pipelines end-to-end. It provides reference-free metrics that assess both the retrieval and generation stages without requiring ground-truth labels — making it practical for production monitoring as well as offline development.

Key features

• Reference-free metrics: faithfulness, answer relevancy, context precision, context recall, context entity recall
• End-to-end dataset evaluation: score entire test sets in one call
• LangChain and LlamaIndex native integrations
• LLM-as-judge architecture — configurable judge model
• CI/CD friendly — JSON output, thresholds, dataset tracking
• Apache-2.0 license

Quick start

pip install ragas

from ragas import evaluate
from ragas.metrics import faithfulness, answer_relevancy, context_recall
from datasets import Dataset

data = {
    "question": ["What year was Python created?"],
    "answer": ["Python was created in 1991."],
    "contexts": [["Python was created by Guido van Rossum and first released in 1991."]],
    "ground_truth": ["1991"]
}

dataset = Dataset.from_dict(data)
result = evaluate(dataset, metrics=[faithfulness, answer_relevancy, context_recall])
print(result)

Install via ai-supply

npx ai-supply add ragas-rag-evaluation

Curated mirror of the open-source RAGAS (Apache-2.0). Get it from the source.