Skip to content
ai-supply.store
DiscoverCategoriesLeaderboardsCommunityAgent APIFAQ
Sign inSign up free
catalog / DevOps & Infra / vLLM
⇄ConnectorDevOps & InfraFree

vLLM

High-throughput, memory-efficient LLM inference engine with PagedAttention and continuous batching.

@ai-supply
Installs892k
⟳ upstream v0.26.0 · updated 2d ago
↗ Source repository
← More DevOps & InfraDevOps & Infra leaderboard →How we grade security →Source ↗
! Grade B · 88/100 · ReviewSecurity assessment
✓No compromise signals48capabilities surfaced1known CVE6of 20 OWASP controls clear
Broad capability surfacePotentially unbounded loopPrompt-injection phrasingVulnerable dependencies
scanned 1d ago·osv · gitleaks · opengrep · picklescan + heuristics·full breakdown in the Security tab ↓

vLLM

vLLM is a fast and easy-to-use library for LLM inference and serving. It achieves state-of-the-art serving throughput through PagedAttention — a novel attention algorithm that effectively manages attention key and value memory — combined with continuous batching of incoming requests and optimized CUDA kernels.

Key Features

  • PagedAttention: near-zero KV cache waste, enabling 24× higher throughput than HuggingFace Transformers
  • Continuous batching: dynamically schedules requests for maximum GPU utilization
  • OpenAI-compatible REST API: drop-in replacement for OpenAI endpoints
  • Quantization support: GPTQ, AWQ, SqueezeLLM, FP8
  • Speculative decoding and chunked prefill
  • Supports 100+ models: Llama, Mistral, Qwen, Phi, Gemma, and more

Quick Start

pip install vllm

# Start OpenAI-compatible server
python -m vllm.entrypoints.openai.api_server \
  --model meta-llama/Llama-3.1-8B-Instruct

# Query it
curl http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model": "meta-llama/Llama-3.1-8B-Instruct", "messages": [{"role": "user", "content": "Hello!"}]}'

Install via ai-supply

npx ai-supply add vllm-high-throughput-inference

Curated mirror of the open-source vLLM (Apache-2.0). Get it from the source.

Rating rank
#1
of 23 in DevOps & Infra
Install rank
#1
of 23 in DevOps & Infra
Security score
88/100 · B
review
Security rank
#10
of 23 in DevOps & Infra
Installs
892k
cat avg 212k
This listing vs category average
Installs
this
cat avg
Security (of 100)
this
cat avg
Adoption trend
See the DevOps & Infra leaderboard →
! Security: Review · 8888/100 · grade Bscanned 1d ago
✓ no compromise signals49 risk-surface · 9/20 OWASP controls flagged

Compromise signals — malicious or tampered code (leaked secrets, backdoors, a dropped executable) — reduce the score, and known dependency CVEs carry a bounded penalty (they warrant review but never QUARANTINE — update the dependency to clear). Other dangerous-by-capability traits are risk surface, expected for some capabilities. Every finding is mapped to its OWASP control below.

What this capability can do · med confidence (static)
⚑ filesystem⚑ shell⚑ network⚑ secretsinstall: script:vllm-project-vllm-1240c74/build_rust.shinstall: script:vllm-project-vllm-1240c74/build_vllm_ppc64le.shinstall: script:vllm-project-vllm-1240c74/docs/pre_run_check.sh
egress → 169.254.169.254, perf.vllm.ai, docs.vllm.ai, huggingface.co, vllm-wheels.s3.amazonaws.com, wheels.vllm.ai, ${s3_bucket}.s3-website-${s3_region}.amazonaws.com, download.pytorch.org +32
auth: bearergithub.comperf.vllm.aidocs.vllm.aivllm-wheels.s3.amazonaws.comwheels.vllm.aipeps.python.org192.168.10.10download.pytorch.orgscope: receive:scope: send:scope: route_namescope: route.matchesscope: quantization.

Findings mapped to the OWASP Top 10 for LLM Applications (2025) and the OWASP Machine Learning Security Top 10. Expand any flagged control for the exact findings — compromise reduces the score; expected/risk-surface do not, except a known CVE, which carries a small bounded penalty (high/critical → Review).

OWASP Top 10 for LLM Applications
⚠LLM01Prompt Injectionhigh
Adversarial instructions embedded in an artifact that hijack a downstream LLM.
•Prompt-injection phrasing — instruction-subversion language detected · vllm-project-vllm-1240c74/docs/getting_started/installation/cpu.arm.inc.md (CWE-77)risk surface
⚠LLM03Supply Chainhigh
Vulnerable/compromised dependencies, models or archives in the artifact.
•Dependency manifest — 1 pip requirements declared · vllm-project-vllm-1240c74/benchmarks/kernels/requirements.txtrisk surface
•Dependency manifest — 6 pip requirements declared · vllm-project-vllm-1240c74/benchmarks/multi_turn/requirements.txtrisk surface
•Vulnerable dependencies — 31 known vulnerabilities in: anyhow@1.0.102, backoff@0.4.0, crossbeam-epoch@0.9.18, instant@0.1.13, paste@1.0.15, proc-macro-error2@2.0.1, pyo3@0.28.3, rand@0.8.5 (CWE-1395)known CVE · -12 pts
⚠LLM05Improper Output Handlinghigh
Code that pipes model/user output into shell, eval, SQL or paths unsafely.
•Suspicious code patterns — OS command execution · vllm-project-vllm-1240c74/.buildkite/scripts/ci-bake-rocm.sh (CWE-78)expected
•Suspicious code patterns — destructive rm -rf / · vllm-project-vllm-1240c74/.buildkite/scripts/hardware_ci/run-amd-test.sh (CWE-78)expected
•Suspicious code patterns — pipe-to-shell install · vllm-project-vllm-1240c74/AGENTS.md (CWE-494)expected
•Suspicious code patterns — pickle deserialization · vllm-project-vllm-1240c74/benchmarks/kernels/graph_machete_bench.py (CWE-502)expected
•Suspicious code patterns — destructive rm -rf /; pipe-to-shell install · vllm-project-vllm-1240c74/docker/Dockerfile (CWE-78)expected
•Suspicious code patterns — dynamic code execution · vllm-project-vllm-1240c74/docs/features/quantization/quark.md (CWE-95)expected
•Suspicious code patterns — environment/secret exfiltration · vllm-project-vllm-1240c74/examples/generate/qwen_1m_offline.py (CWE-200)expected
⚠LLM06Excessive Agencyhigh
Over-broad tool/permission surface or unrestricted egress.
•External endpoints declared — 1 distinct host(s) · vllm-project-vllm-1240c74/.buildkite/check-wheel-size.pyexpected
•Broad capability surface — 3 high-impact capability categories referenced — verify least-privilege · vllm-project-vllm-1240c74/.buildkite/image_build/image_build.sh (CWE-272)risk surface
•Egress to a private/loopback host — 169.254.169.254 · vllm-project-vllm-1240c74/.buildkite/image_build/image_build.sh (CWE-918)expected
•External endpoints declared — 2 distinct host(s) · vllm-project-vllm-1240c74/.buildkite/image_build/image_build.shexpected
•External endpoints declared — 3 distinct host(s) · vllm-project-vllm-1240c74/.buildkite/performance-benchmarks/scripts/run-performance-benchmarks.shexpected
•External endpoints declared — 4 distinct host(s) · vllm-project-vllm-1240c74/.buildkite/scripts/hardware_ci/run-npu-test.shexpected
•Egress to a private/loopback host — 127.0.0.1 · vllm-project-vllm-1240c74/.buildkite/scripts/scheduled_integration_test/deepseek_v2_lite_ep_eplb.sh (CWE-918)expected
•External endpoints declared — 11 distinct host(s) · vllm-project-vllm-1240c74/README.mdexpected
•External endpoints declared — 6 distinct host(s) · vllm-project-vllm-1240c74/RELEASE.mdexpected
•External endpoints declared — 9 distinct host(s) · vllm-project-vllm-1240c74/docker/Dockerfileexpected
•External endpoints declared — 5 distinct host(s) · vllm-project-vllm-1240c74/docker/Dockerfile.rocmexpected
•External endpoints declared — 8 distinct host(s) · vllm-project-vllm-1240c74/docs/README.mdexpected
•Egress to a private/loopback host — 0.0.0.0 · vllm-project-vllm-1240c74/docs/deployment/frameworks/open-webui.md (CWE-918)expected
•External endpoints declared — 7 distinct host(s) · vllm-project-vllm-1240c74/docs/deployment/frameworks/skypilot.mdexpected
•Egress to a private/loopback host — 192.168.0.2, 192.168.0.3 · vllm-project-vllm-1240c74/docs/features/mooncake_connector_usage.md (CWE-918)expected
•External endpoints declared — 12 distinct host(s) · vllm-project-vllm-1240c74/docs/getting_started/installation/gpu.cuda.inc.mdexpected
•External endpoints declared — 14 distinct host(s) · vllm-project-vllm-1240c74/docs/getting_started/quickstart.mdexpected
•Egress to a private/loopback host — 192.168.1.100, 192.168.1.101 · vllm-project-vllm-1240c74/docs/serving/expert_parallel_deployment.md (CWE-918)expected
•External endpoints declared — 21 distinct host(s) · vllm-project-vllm-1240c74/rust/src/chat/src/renderer/deepseek_v32/fixtures/test_input_search_w_date.jsonexpected
⚠LLM07System Prompt Leakagemedium
Secrets, internal hosts or proprietary logic exposed in shipped prompts.
•Internal host / private infrastructure reference — shipped content references a private IP range or internal-only host · vllm-project-vllm-1240c74/.buildkite/scripts/hardware_ci/run-npu-test.sh (CWE-200)expected
⚠LLM10Unbounded Consumptionmedium
Unbounded loops/recursion causing DoS or runaway cost.
Enforced at runtime by the gateway (rate limits + spend caps + size caps); static check flags unbounded loops.
•Potentially unbounded loop — an infinite loop (while True / while(1) / for(;;)) may cause runaway consumption · vllm-project-vllm-1240c74/benchmarks/benchmark_hidden_state_extraction.py (CWE-835)risk surface
§LLM09MisinformationGovernance
Artifacts designed to produce false/deceptive output.
Detectable only by runtime behavioral evaluation; addressed via responsible-use attestation.
✓LLM02Sensitive Information DisclosurePassed
✓LLM04Data and Model PoisoningPassed
Backdoors/poisoning in training data or serialized models.
Behavioral poisoning needs model execution; static check covers unsafe serialization + dataset skew only.
✓LLM08Vector and Embedding WeaknessesPassed
PII or plaintext source leakage in embedding/vector exports.
Embedding inversion/poisoning is largely runtime; static check covers PII in vector exports.
OWASP Machine Learning Security Top 10
⚠ML02Data Poisoninghigh
Poisoned training datasets with triggers or anomalous distributions.
Static check covers trigger phrasing, PII and label skew; full poisoning detection is runtime.
•Prompt-injection phrasing — instruction-subversion language detected · vllm-project-vllm-1240c74/docs/getting_started/installation/cpu.arm.inc.md (CWE-77)risk surface
⚠ML06AI Supply Chainhigh
Compromised PyPI/npm packages, typosquats, unsafe serialized models.
•Dependency manifest — 1 pip requirements declared · vllm-project-vllm-1240c74/benchmarks/kernels/requirements.txtrisk surface
•Dependency manifest — 6 pip requirements declared · vllm-project-vllm-1240c74/benchmarks/multi_turn/requirements.txtrisk surface
•Vulnerable dependencies — 31 known vulnerabilities in: anyhow@1.0.102, backoff@0.4.0, crossbeam-epoch@0.9.18, instant@0.1.13, paste@1.0.15, proc-macro-error2@2.0.1, pyo3@0.28.3, rand@0.8.5 (CWE-1395)known CVE · -12 pts
⚠ML09Output Integrityhigh
Middleware tampering with model outputs in transit.
Gateway enforces TLS + response integrity; static check flags output-rewriting code.
•Suspicious code patterns — OS command execution · vllm-project-vllm-1240c74/.buildkite/scripts/ci-bake-rocm.sh (CWE-78)expected
•Suspicious code patterns — destructive rm -rf / · vllm-project-vllm-1240c74/.buildkite/scripts/hardware_ci/run-amd-test.sh (CWE-78)expected
•Suspicious code patterns — pipe-to-shell install · vllm-project-vllm-1240c74/AGENTS.md (CWE-494)expected
•Suspicious code patterns — pickle deserialization · vllm-project-vllm-1240c74/benchmarks/kernels/graph_machete_bench.py (CWE-502)expected
•Suspicious code patterns — destructive rm -rf /; pipe-to-shell install · vllm-project-vllm-1240c74/docker/Dockerfile (CWE-78)expected
•Suspicious code patterns — dynamic code execution · vllm-project-vllm-1240c74/docs/features/quantization/quark.md (CWE-95)expected
•Suspicious code patterns — environment/secret exfiltration · vllm-project-vllm-1240c74/examples/generate/qwen_1m_offline.py (CWE-200)expected
§ML01Input Manipulation (Adversarial)Governance
Models vulnerable to adversarial perturbations.
Requires runtime robustness evaluation; addressed via publisher robustness attestation.
§ML03Model InversionGovernance
Training data reconstructable from a model's outputs.
Runtime/evaluation property; addressed via model-card data-provenance + DP attestation.
§ML04Membership InferenceGovernance
Determining whether a record was in the training set.
Runtime/evaluation property; addressed via overfitting disclosure + DP attestation.
§ML08Model SkewingGovernance
Models trained on skewed data producing biased output.
Requires fairness evaluation; addressed via model-card bias/limitations disclosure.
✓ML05Model TheftPassed
Unlicensed re-distribution / license-incompatible derivatives.
Static check verifies license declaration; extraction throttling is runtime.
✓ML07Transfer Learning AttackPassed
Backdoored base models / LoRA adapters propagating to derivatives.
Backdoor detection needs behavioral probing; static check covers unsafe serialization + provenance.
✓ML10Model Poisoning (Weights)Passed
Tampered model weight files; integrity must be verifiable.
Static check enforces safe formats + records a content hash for downstream verification.
Other findings (49) · hygiene / uncategorized
•Suspicious network references — raw IP URL (6 URLs) · vllm-project-vllm-1240c74/.buildkite/image_build/image_build.shexpected
•Suspicious network references — suspicious TLD (8 URLs) · vllm-project-vllm-1240c74/.buildkite/scripts/hardware_ci/run-npu-test.shexpected
•Suspicious network references — suspicious TLD (4 URLs) · vllm-project-vllm-1240c74/.buildkite/scripts/run-rust-frontend-cargo-ci.shexpected
•Suspicious network references — raw IP URL (2 URLs) · vllm-project-vllm-1240c74/.buildkite/scripts/scheduled_integration_test/deepseek_v2_lite_ep_eplb.shexpected
•Suspicious network references — raw IP URL (3 URLs) · vllm-project-vllm-1240c74/.buildkite/scripts/scheduled_integration_test/deepseek_v2_lite_prefetch_offload.shexpected
•Unrecognized file type — '.env' is not on the allowlist · vllm-project-vllm-1240c74/.buildkite/scripts/tpu/config_v6e_1.envrisk surface
•Unrecognized file type — '.clang-format' is not on the allowlist · vllm-project-vllm-1240c74/.clang-formatrisk surface
•Unrecognized file type — '.coveragerc' is not on the allowlist · vllm-project-vllm-1240c74/.coveragercrisk surface
•Unrecognized file type — '.dockerignore' is not on the allowlist · vllm-project-vllm-1240c74/.dockerignorerisk surface
•Unrecognized file type — '.git-blame-ignore-revs' is not on the allowlist · vllm-project-vllm-1240c74/.git-blame-ignore-revsrisk surface
•Unrecognized file type — '.?' is not on the allowlist · vllm-project-vllm-1240c74/.github/CODEOWNERSrisk surface
•Unrecognized file type — '.gitignore' is not on the allowlist · vllm-project-vllm-1240c74/.gitignorerisk surface
•Unrecognized file type — '.shellcheckrc' is not on the allowlist · vllm-project-vllm-1240c74/.shellcheckrcrisk surface
•Unrecognized file type — '.in' is not on the allowlist · vllm-project-vllm-1240c74/MANIFEST.inrisk surface
•Suspicious network references — raw IP URL (1 URLs) · vllm-project-vllm-1240c74/benchmarks/kv_cache_watermark.shexpected
•Unrecognized file type — '.cmake' is not on the allowlist · vllm-project-vllm-1240c74/cmake/cpu_extension.cmakerisk surface
•Unrecognized file type — '.h' is not on the allowlist · vllm-project-vllm-1240c74/csrc/attention/attention_dtypes.hrisk surface
•Unrecognized file type — '.cuh' is not on the allowlist · vllm-project-vllm-1240c74/csrc/attention/attention_generic.cuhrisk surface
•Unrecognized file type — '.hpp' is not on the allowlist · vllm-project-vllm-1240c74/csrc/core/batch_invariant.hpprisk surface
•Unrecognized file type — '.cpp' is not on the allowlist · vllm-project-vllm-1240c74/csrc/cpu/activation.cpprisk surface
•Unrecognized file type — '.cu' is not on the allowlist · vllm-project-vllm-1240c74/csrc/custom_quickreduce.curisk surface
•Unrecognized file type — '.inl' is not on the allowlist · vllm-project-vllm-1240c74/csrc/libtorch_stable/moe/permute_unpermute_kernels/moe_permute_unpermute_kernel.inlrisk surface
•Unrecognized file type — '.cpu' is not on the allowlist · vllm-project-vllm-1240c74/docker/Dockerfile.cpurisk surface
•Unrecognized file type — '.ppc64le' is not on the allowlist · vllm-project-vllm-1240c74/docker/Dockerfile.ppc64lerisk surface
•Unrecognized file type — '.rocm' is not on the allowlist · vllm-project-vllm-1240c74/docker/Dockerfile.rocmrisk surface
•Unrecognized file type — '.rocm_base' is not on the allowlist · vllm-project-vllm-1240c74/docker/Dockerfile.rocm_baserisk surface
•Unrecognized file type — '.s390x' is not on the allowlist · vllm-project-vllm-1240c74/docker/Dockerfile.s390xrisk surface
•Unrecognized file type — '.tpu' is not on the allowlist · vllm-project-vllm-1240c74/docker/Dockerfile.tpurisk surface
•Unrecognized file type — '.xpu' is not on the allowlist · vllm-project-vllm-1240c74/docker/Dockerfile.xpurisk surface
•Unrecognized file type — '.hcl' is not on the allowlist · vllm-project-vllm-1240c74/docker/ci-rocm.hclrisk surface
•Suspicious network references — suspicious TLD (20 URLs) · vllm-project-vllm-1240c74/docs/benchmarking/cli.mdexpected
•Suspicious network references — raw IP URL (21 URLs) · vllm-project-vllm-1240c74/docs/contributing/README.mdexpected
•Suspicious network references — raw IP URL (4 URLs) · vllm-project-vllm-1240c74/docs/deployment/frameworks/open-webui.mdexpected
•Suspicious network references — raw IP URL (7 URLs) · vllm-project-vllm-1240c74/docs/deployment/k8s.mdexpected
•Suspicious network references — raw IP URL (59 URLs) · vllm-project-vllm-1240c74/docs/design/metrics.mdexpected
•Suspicious network references — raw IP URL (5 URLs) · vllm-project-vllm-1240c74/docs/features/mooncake_connector_usage.mdexpected
•Suspicious network references — raw IP URL (9 URLs) · vllm-project-vllm-1240c74/docs/features/structured_outputs.mdexpected
•Suspicious network references — suspicious TLD (35 URLs) · vllm-project-vllm-1240c74/docs/getting_started/quickstart.mdexpected
•Suspicious network references — raw IP URL (12 URLs) · vllm-project-vllm-1240c74/docs/models/pooling_models/embed.mdexpected
•Suspicious network references — raw IP URL (11 URLs) · vllm-project-vllm-1240c74/docs/models/pooling_models/scoring.mdexpected
•Suspicious network references — suspicious TLD (23 URLs) · vllm-project-vllm-1240c74/docs/models/supported_models.mdexpected
•Unrecognized file type — '.helmignore' is not on the allowlist · vllm-project-vllm-1240c74/examples/deployment/chart-helm/.helmignorerisk surface
•Unrecognized file type — '.tpl' is not on the allowlist · vllm-project-vllm-1240c74/examples/deployment/chart-helm/templates/_helpers.tplrisk surface
•Unrecognized file type — '.jsonl' is not on the allowlist · vllm-project-vllm-1240c74/examples/features/openai_batch/openai_example_batch.jsonlrisk surface
•Unrecognized file type — '.jinja' is not on the allowlist · vllm-project-vllm-1240c74/examples/pooling/embed/template/dse_qwen2_vl.jinjarisk surface
•Suspicious network references — raw IP URL (8 URLs) · vllm-project-vllm-1240c74/examples/tool_calling/openai_responses_client_with_mcp_tools.pyexpected
•Unrecognized file type — '.gitattributes' is not on the allowlist · vllm-project-vllm-1240c74/rust/.gitattributesrisk surface
•Unrecognized file type — '.proto' is not on the allowlist · vllm-project-vllm-1240c74/rust/proto/vllm_grpc.protorisk surface
•Suspicious network references — raw IP URL (37 URLs) · vllm-project-vllm-1240c74/rust/src/bench/README.mdexpected
✔ verified source · pinned vllm-project-vllm-1240c74
Check against a policy

The same gate an agent runs before installing (POST /api/v1/trust/vllm-high-throughput-inference/check). Click a policy:

Consume vLLM programmatically. Authenticate with an API key or session — see Authorize an agent.

# Agents: CHECK BEFORE YOU INSTALL (no auth) — score, grade, level, capability manifest
curl https://ai-supply.store/api/v1/trust/vllm-high-throughput-inference

# Gate against your org policy (returns { pass, violations })
curl -X POST https://ai-supply.store/api/v1/trust/vllm-high-throughput-inference/check \
  -H "Content-Type: application/json" \
  -d '{"minGrade":"B","denyPermissions":["shell"],"denyUnknownEgress":true}'

# CLI
npx ai-supply add vllm-high-throughput-inference

# REST (install → download)
curl -X POST https://ai-supply.store/api/v1/listings/vllm-high-throughput-inference/install \
  -H "Authorization: Bearer $AIM_KEY"

# MCP tool
install_listing({ "slug": "vllm-high-throughput-inference" })
OpenAPI spec →
vlatest
! Security: Review · 881mo ago

Curated mirror — latest upstream source. See the repository for tagged releases.

Sign in and install this listing to leave a review.

More from @ai-supply

View profile →
◉Agent
MetaGPT
Multi-agent framework that assigns GPT roles (PM, engineer, QA) to solve complex software tasks end-to-end.
↓ 1.0M
⇄Connector
Meilisearch
Lightning-fast open-source search engine with typo-tolerance, semantic hybrid search, and sub-50ms response times.
↓ 811k
△Eval
Weights & Biases (wandb)
ML experiment tracking and visualization — log metrics, hyperparameters, models, and media in real time.
↓ 784k
◇MCP server
GitHub MCP Server
Official GitHub MCP server — give your AI agent full read/write access to repos, issues, PRs, and actions.
↓ 771k
ai-supply.store

Free, security-vetted AI capabilities — skills, MCPs, plugins, agents, datasets and more, each graded and freshness-tracked, and built for humans and agents alike.

api · v3.1status · all green
Contact
support@ai-supply.storesecurity@ai-supply.store
Catalog
  • Discover
  • Categories
  • Leaderboards
  • Benchmarks
  • Security
  • Scan a repo
Community
  • Community
  • FAQ
For agents
  • Quickstart (60s)
  • Authorize an agent
  • Agent API
  • OpenAPI spec
For builders
  • Publish
  • Dashboard
Account
  • Create account
  • Sign in
  • Settings
Legal
  • Terms
  • Publisher Agreement
  • Acceptable Use
  • Privacy