Skip to content
ai-supply.store
DiscoverCategoriesLeaderboardsCommunityAgent APIFAQ
Sign inSign up free
catalog / Coding / Schemathesis
⌬WorkflowCodingFree

Schemathesis

Generate and run property-based tests against any OpenAPI or GraphQL API to automatically surface crashes and spec violations.

@ai-supply
Installs27k
⟳ upstream v4.22.3 · updated 3d ago
↗ Source repository
← More CodingCoding leaderboard →How we grade security →Source ↗

Schemathesis

Schemathesis reads your OpenAPI or GraphQL schema and automatically generates test cases that probe every endpoint with valid and adversarial inputs. Built on Hypothesis, it uses property-based testing to find responses that violate the schema, crash the server, or expose edge-case bugs — no hand-written test suite required.

Key features

  • Auto-derives thousands of test cases straight from your API schema
  • Property-based fuzzing that shrinks failures to minimal reproductions
  • Stateful testing that chains operations to find sequence-dependent bugs
  • Runs as a CLI or pytest integration; fits CI pipelines
  • Conformance checks: status codes, schemas, content types, and headers

A focused test-generation tool for API teams: point it at a spec and it manufactures the coverage, catching contract regressions before users do.

Curated mirror of the open-source Schemathesis (MIT). Get it from the source.

Rating rank
#1
of 20 in Coding
Install rank
#18
of 20 in Coding
Security score
100/100 · A
safe
Security rank
#1
of 20 in Coding
Installs
27k
cat avg 217k
This listing vs category average
Installs
this
cat avg
Security (of 100)
this
cat avg
Adoption trend
See the Coding leaderboard →
✓ Security: Safe · 100100/100 · grade Ascanned 1h ago

Only compromise signals — malicious or tampered code (leaked secrets, backdoors, path traversal, a dropped executable) — reduce the score. Dangerous-by-capability traits (shell, network, injection strings, pickle) are shown as risk surface: expected for some capabilities — a security tool ships offensive code on purpose — so they do not sink the grade.

Compromise signals
None — no malicious or tampered code detected in the scanned source.
What this capability can do · med confidence (static)
⚑ filesystem⚑ shell⚑ network⚑ secrets
egress → api.example.com, discord.gg, docs.python.org, schemathesis.readthedocs.io, no-color.org, spec.openapis.org, swagger.io
Risk surface (42)
External endpoints declaredlowschemathesis-schemathesis-9cfdf69/.github/ISSUE_TEMPLATE/bug_report.md
2 distinct host(s)
External endpoints declaredlowschemathesis-schemathesis-9cfdf69/.github/ISSUE_TEMPLATE/trophy-submission.yml
1 distinct host(s)
Broad capability surfacelowschemathesis-schemathesis-9cfdf69/.github/workflows/build.yml
3 high-impact capability categories referenced — verify least-privilege
Suspicious network referencesmediumschemathesis-schemathesis-9cfdf69/.github/workflows/example-build.yml
raw IP URL (1 URLs)
Egress to a private/loopback hosthighschemathesis-schemathesis-9cfdf69/.github/workflows/example-build.yml
127.0.0.1
Path traversal sequencesmediumschemathesis-schemathesis-9cfdf69/CHANGELOG.md
'../' present in content or name
Suspicious network referencesmediumschemathesis-schemathesis-9cfdf69/CHANGELOG.md
raw IP URL (1231 URLs)
Broad capability surfacelowschemathesis-schemathesis-9cfdf69/CHANGELOG.md
4 high-impact capability categories referenced — verify least-privilege
Egress to a private/loopback hosthighschemathesis-schemathesis-9cfdf69/CHANGELOG.md
127.0.0.1, 0.0.0.0
External endpoints declaredmediumschemathesis-schemathesis-9cfdf69/CHANGELOG.md
12 distinct host(s)
External endpoints declaredlowschemathesis-schemathesis-9cfdf69/CITATION.cff
3 distinct host(s)
Suspicious network referencesmediumschemathesis-schemathesis-9cfdf69/CONTRIBUTING.md
raw IP URL (10 URLs)
External endpoints declaredlowschemathesis-schemathesis-9cfdf69/CONTRIBUTING.md
5 distinct host(s)
Suspicious code patternshighschemathesis-schemathesis-9cfdf69/Dockerfile
destructive rm -rf /
Suspicious code patternsmediumschemathesis-schemathesis-9cfdf69/benches/memory/measure.py
OS command execution
Suspicious network referencesmediumschemathesis-schemathesis-9cfdf69/benches/response_conformance.py
raw IP URL (5 URLs)
External endpoints declaredlowschemathesis-schemathesis-9cfdf69/docs/explanations/examples.md
4 distinct host(s)
Suspicious network referencesmediumschemathesis-schemathesis-9cfdf69/docs/explanations/pytest.md
raw IP URL (3 URLs)
Embedded credentialshighexpected for this capabilityschemathesis-schemathesis-9cfdf69/docs/guides/auth.md
found: hardcoded credential
Suspicious network referencesmediumschemathesis-schemathesis-9cfdf69/docs/guides/crash-reproduction.md
raw IP URL (4 URLs)
Internal host / private infrastructure referencemediumschemathesis-schemathesis-9cfdf69/docs/guides/docker.md
shipped content references a private IP range or internal-only host
Embedded credentialshighexpected for this capabilityschemathesis-schemathesis-9cfdf69/docs/reference/configuration.md
found: credentials in URL
External endpoints declaredlowschemathesis-schemathesis-9cfdf69/docs/reference/configuration.md
7 distinct host(s)
External endpoints declaredmediumschemathesis-schemathesis-9cfdf69/docs/resources.md
19 distinct host(s)
Suspicious network referencesmediumschemathesis-schemathesis-9cfdf69/docs/tutorials/cli.md
raw IP URL (15 URLs)
External endpoints declaredlowschemathesis-schemathesis-9cfdf69/docs/tutorials/cli.md
8 distinct host(s)
Suspicious network referencesmediumschemathesis-schemathesis-9cfdf69/docs/tutorials/pytest.md
raw IP URL (11 URLs)
Potentially unbounded loopmediumschemathesis-schemathesis-9cfdf69/src/schemathesis/cli/commands/run/handlers/crashes.py
an infinite loop (while True / while(1) / for(;;)) may cause runaway consumption
Suspicious code patternsmediumschemathesis-schemathesis-9cfdf69/src/schemathesis/core/deserialization.py
unsafe yaml.load
Suspicious network referencesmediumschemathesis-schemathesis-9cfdf69/src/schemathesis/schemas.py
raw IP URL (2 URLs)
Embedded credentialshighexpected for this capabilityschemathesis-schemathesis-9cfdf69/test/cli/cert.pem
found: private key
Suspicious network referencesmediumschemathesis-schemathesis-9cfdf69/test/cli/test_commands.py
raw IP URL (33 URLs)
Suspicious network referencesmediumschemathesis-schemathesis-9cfdf69/test/cli/test_config_file.py
raw IP URL (6 URLs)
Suspicious code patternsmediumschemathesis-schemathesis-9cfdf69/test/cli/test_warnings.py
dynamic code execution
Suspicious network referencesmediumschemathesis-schemathesis-9cfdf69/test/code_samples/test_curl.py
raw IP URL (12 URLs)
Suspicious network referencesmediumschemathesis-schemathesis-9cfdf69/test/core/test_sanitizing_output.py
raw IP URL (16 URLs)
Suspicious network referencesmediumschemathesis-schemathesis-9cfdf69/test/coverage/test_phase.py
raw IP URL (31 URLs)
Egress to a private/loopback hosthighschemathesis-schemathesis-9cfdf69/test/pytest/test_main.py
192.168.1.4, 127.0.0.1
Suspicious network referencesmediumschemathesis-schemathesis-9cfdf69/test/reporting/test_crashes.py
raw IP URL (8 URLs)
Egress to a private/loopback hosthighschemathesis-schemathesis-9cfdf69/test/specs/graphql/test_basic.py
0.0.0.0, 127.0.0.1
Suspicious network referencesmediumschemathesis-schemathesis-9cfdf69/test/specs/openapi/adapter/test_servers.py
raw IP URL (32 URLs)
External endpoints declaredlowschemathesis-schemathesis-9cfdf69/test/specs/openapi/adapter/test_servers.py
10 distinct host(s)
✔ verified source · pinned schemathesis-schemathesis-9cfdf69 · changed since last scan · +egress api.example.com, discord.gg, docs.python.org, schemathesis.readthedocs.io, no-color.org, spec.openapis.org, swagger.io
OWASP AI control mapping
9passed
6flagged
0runtime-enforced
5governance

Evaluated against the OWASP Top 10 for LLM Applications (2025) and the OWASP Machine Learning Security Top 10. Expand any control to see the findings.

OWASP Top 10 for LLM Applications
✓LLM01Prompt InjectionPassed
⚠LLM02Sensitive Information Disclosurehigh
Secrets, credentials or PII shipped inside the artifact.
•Embedded credentials — found: hardcoded credential (CWE-798)
•Embedded credentials — found: credentials in URL (CWE-798)
•Embedded credentials — found: private key (CWE-798)
✓LLM03Supply ChainPassed
✓LLM04Data and Model PoisoningPassed
Backdoors/poisoning in training data or serialized models.
Behavioral poisoning needs model execution; static check covers unsafe serialization + dataset skew only.
⚠LLM05Improper Output Handlinghigh
Code that pipes model/user output into shell, eval, SQL or paths unsafely.
•Path traversal sequences — '../' present in content or name (CWE-22)
•Suspicious code patterns — destructive rm -rf / (CWE-78)
•Suspicious code patterns — OS command execution (CWE-78)
•Suspicious code patterns — unsafe yaml.load (CWE-502)
•Suspicious code patterns — dynamic code execution (CWE-95)
⚠LLM06Excessive Agencyhigh
Over-broad tool/permission surface or unrestricted egress.
•External endpoints declared — 2 distinct host(s)
•External endpoints declared — 1 distinct host(s)
•Broad capability surface — 3 high-impact capability categories referenced — verify least-privilege (CWE-272)
•Egress to a private/loopback host — 127.0.0.1 (CWE-918)
•Broad capability surface — 4 high-impact capability categories referenced — verify least-privilege (CWE-272)
•Egress to a private/loopback host — 127.0.0.1, 0.0.0.0 (CWE-918)
•External endpoints declared — 12 distinct host(s)
•External endpoints declared — 3 distinct host(s)
•External endpoints declared — 5 distinct host(s)
•External endpoints declared — 4 distinct host(s)
•External endpoints declared — 7 distinct host(s)
•External endpoints declared — 19 distinct host(s)
•External endpoints declared — 8 distinct host(s)
•Egress to a private/loopback host — 192.168.1.4, 127.0.0.1 (CWE-918)
•Egress to a private/loopback host — 0.0.0.0, 127.0.0.1 (CWE-918)
•External endpoints declared — 10 distinct host(s)
⚠LLM07System Prompt Leakagehigh
Secrets, internal hosts or proprietary logic exposed in shipped prompts.
•Embedded credentials — found: hardcoded credential (CWE-798)
•Internal host / private infrastructure reference — shipped content references a private IP range or internal-only host (CWE-200)
•Embedded credentials — found: credentials in URL (CWE-798)
•Embedded credentials — found: private key (CWE-798)
✓LLM08Vector and Embedding WeaknessesPassed
PII or plaintext source leakage in embedding/vector exports.
Embedding inversion/poisoning is largely runtime; static check covers PII in vector exports.
§LLM09MisinformationGovernance
Artifacts designed to produce false/deceptive output.
Detectable only by runtime behavioral evaluation; addressed via responsible-use attestation.
⚠LLM10Unbounded Consumptionmedium
Unbounded loops/recursion causing DoS or runaway cost.
Enforced at runtime by the gateway (rate limits + spend caps + size caps); static check flags unbounded loops.
•Potentially unbounded loop — an infinite loop (while True / while(1) / for(;;)) may cause runaway consumption (CWE-835)
OWASP Machine Learning Security Top 10
§ML01Input Manipulation (Adversarial)Governance
Models vulnerable to adversarial perturbations.
Requires runtime robustness evaluation; addressed via publisher robustness attestation.
✓ML02Data PoisoningPassed
Poisoned training datasets with triggers or anomalous distributions.
Static check covers trigger phrasing, PII and label skew; full poisoning detection is runtime.
§ML03Model InversionGovernance
Training data reconstructable from a model's outputs.
Runtime/evaluation property; addressed via model-card data-provenance + DP attestation.
§ML04Membership InferenceGovernance
Determining whether a record was in the training set.
Runtime/evaluation property; addressed via overfitting disclosure + DP attestation.
✓ML05Model TheftPassed
Unlicensed re-distribution / license-incompatible derivatives.
Static check verifies license declaration; extraction throttling is runtime.
✓ML06AI Supply ChainPassed
✓ML07Transfer Learning AttackPassed
Backdoored base models / LoRA adapters propagating to derivatives.
Backdoor detection needs behavioral probing; static check covers unsafe serialization + provenance.
§ML08Model SkewingGovernance
Models trained on skewed data producing biased output.
Requires fairness evaluation; addressed via model-card bias/limitations disclosure.
⚠ML09Output Integrityhigh
Middleware tampering with model outputs in transit.
Gateway enforces TLS + response integrity; static check flags output-rewriting code.
•Path traversal sequences — '../' present in content or name (CWE-22)
•Suspicious code patterns — destructive rm -rf / (CWE-78)
•Suspicious code patterns — OS command execution (CWE-78)
•Suspicious code patterns — unsafe yaml.load (CWE-502)
•Suspicious code patterns — dynamic code execution (CWE-95)
✓ML10Model Poisoning (Weights)Passed
Tampered model weight files; integrity must be verifiable.
Static check enforces safe formats + records a content hash for downstream verification.

Consume Schemathesis programmatically. Authenticate with an API key or session — see Authorize an agent.

# Agents: CHECK BEFORE YOU INSTALL (no auth) — score, grade, level, capability manifest
curl https://ai-supply.store/api/v1/trust/schemathesis-api-test-generator

# Gate against your org policy (returns { pass, violations })
curl -X POST https://ai-supply.store/api/v1/trust/schemathesis-api-test-generator/check \
  -H "Content-Type: application/json" \
  -d '{"minGrade":"B","denyPermissions":["shell"],"denyUnknownEgress":true}'

# CLI
npx ai-supply add schemathesis-api-test-generator

# REST (install → download)
curl -X POST https://ai-supply.store/api/v1/listings/schemathesis-api-test-generator/install \
  -H "Authorization: Bearer $AIM_KEY"

# MCP tool
install_listing({ "slug": "schemathesis-api-test-generator" })
OpenAPI spec →
vlatest
✓ Security: Safe · 1001d ago

Curated mirror — latest upstream source. See the repository for tagged releases.

Sign in and install this listing to leave a review.

More from @ai-supply

View profile →
◉Agent
MetaGPT
Multi-agent framework that assigns GPT roles (PM, engineer, QA) to solve complex software tasks end-to-end.
↓ 1.0M
◐Model
llama.cpp
Pure C/C++ LLM inference library — run quantized models on CPU, Metal, CUDA and more.
↓ 1.0M★ 4.5
⇄Connector
vLLM
High-throughput, memory-efficient LLM inference engine with PagedAttention and continuous batching.
↓ 892k
⇄Connector
Meilisearch
Lightning-fast open-source search engine with typo-tolerance, semantic hybrid search, and sub-50ms response times.
↓ 811k
ai-supply.store

Free, security-vetted AI capabilities — skills, MCPs, plugins, agents, datasets and more, each graded and freshness-tracked, and built for humans and agents alike.

api · v3.1status · all green
Contact
support@ai-supply.storesecurity@ai-supply.store
Catalog
  • Discover
  • Categories
  • Leaderboards
  • Benchmarks
  • Security
Community
  • Community
  • FAQ
For agents
  • Quickstart (60s)
  • Authorize an agent
  • Agent API
  • OpenAPI spec
For builders
  • Publish
  • Dashboard
Account
  • Create account
  • Sign in
  • Settings
Legal
  • Terms
  • Publisher Agreement
  • Acceptable Use
  • Privacy