Prompt flow
Microsoft's open-source toolkit for building, evaluating, and deploying high-quality LLM applications end-to-end.
Prompt flow
Prompt flow is a suite of development tools designed to streamline the entire LLM application development lifecycle — from ideation and prototyping to testing, evaluation, and production deployment. It provides a visual graph of flow nodes, built-in evaluators, and CI/CD integration.
Key Features
- Visual DAG editor: Build LLM flows as connected nodes (LLM calls, Python functions, tools) with a drag-and-drop graph
- Flex flows: Python-first authoring with full control and no YAML required
- Batch evaluation: Run your flow over datasets and compute quality metrics automatically
- Built-in evaluators: Groundedness, relevance, coherence, fluency, and custom metric support
- Tracing: OpenTelemetry-compatible distributed tracing for debugging complex multi-step flows
- CI/CD ready: CLI + SDK for integrating quality gates into GitHub Actions or Azure Pipelines
Quick Start
pip install promptflow promptflow-tools
# Create a new flow
pf flow init --flow my-chat-flow --type chat
# Run the flow
pf flow test --flow my-chat-flow --inputs question="What is LLM?"
# Batch evaluate
pf run create --flow my-chat-flow --data data.jsonl
Add to ai-supply
npx ai-supply add promptflow-llm-pipeline
Curated mirror of the open-source Prompt flow (MIT). Get it from the source.
Compromise signals — malicious or tampered code (leaked secrets, backdoors, a dropped executable) — reduce the score, and known dependency CVEs carry a bounded penalty (they warrant review but never QUARANTINE — update the dependency to clear). Other dangerous-by-capability traits are risk surface, expected for some capabilities. Every finding is mapped to its OWASP control below.
Findings mapped to the OWASP Top 10 for LLM Applications (2025) and the OWASP Machine Learning Security Top 10. Expand any flagged control for the exact findings — compromise reduces the score; expected/risk-surface do not, except a known CVE, which carries a small bounded penalty (high/critical → Review).
The same gate an agent runs before installing (POST /api/v1/trust/promptflow-llm-pipeline/check). Click a policy:
Consume Prompt flow programmatically. Authenticate with an API key or session — see Authorize an agent.
# Agents: CHECK BEFORE YOU INSTALL (no auth) — score, grade, level, capability manifest
curl https://ai-supply.store/api/v1/trust/promptflow-llm-pipeline
# Gate against your org policy (returns { pass, violations })
curl -X POST https://ai-supply.store/api/v1/trust/promptflow-llm-pipeline/check \
-H "Content-Type: application/json" \
-d '{"minGrade":"B","denyPermissions":["shell"],"denyUnknownEgress":true}'
# CLI
npx ai-supply add promptflow-llm-pipeline
# REST (install → download)
curl -X POST https://ai-supply.store/api/v1/listings/promptflow-llm-pipeline/install \
-H "Authorization: Bearer $AIM_KEY"
# MCP tool
install_listing({ "slug": "promptflow-llm-pipeline" })OpenAPI spec →Curated mirror — latest upstream source. See the repository for tagged releases.