End-to-end ML lifecycle platform — experiment tracking, model registry, serving, and LLM evaluation.

ML experiment tracking and visualization — log metrics, hyperparameters, models, and media in real time.

ProjectDiscovery's fast, template-driven vulnerability scanner with 9,000+ community templates for web apps, APIs, cloud, and AI service endpoints.

Data quality framework for defining, testing, and documenting expectations about your data pipelines.

Apache-licensed multi-cloud security assessment tool covering 500+ checks across AWS, Azure, GCP, and Kubernetes, including AI service misconfigurations.

Open-source ML and LLM observability framework for evaluating, monitoring, and testing AI system quality.

EleutherAI's MIT-licensed unified benchmark suite — the de-facto standard for evaluating language models across 200+ tasks.

Self-hosted, open-source ML training metadata tracker with a powerful exploratory web UI.

MIT-licensed framework for evaluating LLMs and AI systems — build custom evals, run model comparisons, log results.

NVIDIA's open-source LLM vulnerability scanner that probes language models for prompt injection, jailbreaks, hallucinations, and more.

OpenAI's MIT-licensed suite of 16 procedurally-generated 2D game environments for measuring generalization in reinforcement learning agents.

Apache-2.0 LLM testing framework — pytest-style unit tests for RAG pipelines, chatbots, and LLM outputs.

Microsoft's open-source AI red-teaming toolkit for systematically finding risks in generative AI systems through automated adversarial probing.

Apache-2.0 RAG evaluation framework — faithfulness, answer relevancy, context recall, and more in one pip install.

LLM eval + red-teaming framework — test prompts and models against custom assertions, compare providers, and catch regressions in CI.

Stanford CRFM's reproducible, multi-metric benchmark framework for evaluating any foundation model.

Open-source LLM evaluation and observability platform — trace, test, and monitor LLM apps end-to-end.

Maintained fork of Quantopian's pyfolio providing comprehensive risk and return analytics, tear sheets, and performance attribution for quantitative strategies.

Multi-task benchmark for legal NLP with 7 datasets covering EURLEX classification, contract clause labeling, court judgement prediction, and more.

Google DeepMind's suite of 50+ multi-agent social RL scenarios testing cooperation, competition, and generalization.