garak — LLM Vulnerability Scanner

garak — LLM Vulnerability Scanner

garak (Generative AI Red-teaming & Assessment Kit) is NVIDIA's open-source security evaluation framework for large language models. It systematically probes LLMs for a wide catalogue of weaknesses — prompt injection, jailbreaks, toxic content generation, hallucinations, data leakage, encoding exploits, and more — making it the de-facto starting point for any AI red-team engagement.

Key Features

• 100+ built-in probes covering OWASP LLM Top-10 and beyond
• Pluggable generator backends: OpenAI, HuggingFace, Ollama, REST, and local models
• HTML + JSONL report output for audit trails
• Extensible probe / detector / harness architecture
• CI-ready — exit codes reflect pass/fail thresholds

Quick Start

pip install garak
# Scan an OpenAI model for prompt-injection
garak --model_type openai --model_name gpt-4o --probes promptinject
# Full vulnerability sweep
garak --model_type huggingface --model_name meta-llama/Llama-3-8B --probes all

npx ai-supply add garak-llm-vulnerability-scanner

Curated mirror of the open-source garak (Apache-2.0). Get it from the source.