Prefect

Prefect is the modern open-source workflow orchestration platform. Decorate any Python function as a @flow or @task, and Prefect handles scheduling, retries, observability, logging, and deployment — with a beautiful UI and cloud hosting option.

Key features

• Pythonic API — @flow and @task decorators require zero schema changes to existing code
• Automatic retries — configure retry counts and wait strategies per task
• Scheduling — cron, interval, or event-driven triggers
• Prefect UI — real-time flow run dashboard with logs, state history, and artifact tracking
• Deployments — package flows and deploy to Prefect Cloud, Kubernetes, ECS, or local agents
• Integrations — 100+ task libraries for AWS, GCP, dbt, Snowflake, Slack, and more

Quick start

npx ai-supply add prefect-workflow-orchestration

# Or install directly
pip install prefect

from prefect import flow, task

@task(retries=3)
def fetch_data(url: str) -> dict:
    import httpx
    return httpx.get(url).json()

@task
def process(data: dict) -> str:
    return f"Got {len(data)} keys"

@flow(log_prints=True)
def my_pipeline(url: str):
    data = fetch_data(url)
    result = process(data)
    print(result)

if __name__ == "__main__":
    my_pipeline("https://api.github.com")

# Deploy and schedule
prefect deploy --name my-pipeline --cron "0 * * * *"

Curated mirror of the open-source Prefect project (Apache-2.0). Install upstream from the repository.