Counterfit — ML Model Security Testing CLI

Counterfit — ML Model Security Testing CLI

Counterfit is a Microsoft Azure open-source CLI that provides a generic automation layer for assessing AI/ML model security. It wraps popular attack libraries (ART, TextAttack, Augly) behind a single interface so red teams can probe any model — whether it's a REST endpoint, a local PyTorch model, or a cloud API.

Key features

• Unified interface over 20+ attack algorithms (evasion, poisoning, model extraction, inference)
• Works against black-box REST endpoints — no source code required
• Replay attack logs for reproducible security reports
• Out-of-the-box target adapters for image classifiers and NLP models
• Built-in logging to Azure Monitor or local JSON

Quick start

pip install counterfit
cf # launch interactive CLI
# Inside the CLI:
list targets
set_target common-cartpole
list attacks
set_attack art-boundary
run

npx ai-supply add counterfit-ml-security-cli

Curated mirror of the open-source Counterfit (MIT). Get it from the source.