DevOps & Infra

High-throughput, memory-efficient LLM inference engine with PagedAttention and continuous batching.

End-to-end ML lifecycle platform — experiment tracking, model registry, serving, and LLM evaluation.

ML experiment tracking and visualization — log metrics, hyperparameters, models, and media in real time.

Distributed Python framework for scaling ML workloads — training, serving, hyperparameter tuning, and pipelines.

Apache-2.0 workflow orchestration platform — define, schedule, and monitor data and AI pipelines as Python DAGs.

Build, ship, and scale AI services — unified framework from local development to production Kubernetes.

Ship an entire LLM as a single self-contained executable — one file, every OS, no install needed.

Fast LLM serving framework with RadixAttention KV cache reuse and structured output (JSON/regex) natively.

Production LLM serving platform by BentoML — deploy any open-source LLM with one command.

Open-source ML and LLM observability framework for evaluating, monitoring, and testing AI system quality.

Modern open-source data pipeline builder with an interactive code editor, real-time execution, and built-in ML integrations.

Self-hosted, open-source ML training metadata tracker with a powerful exploratory web UI.

OpenTelemetry-based observability SDK for LLM applications — traces, metrics, and logs for any AI framework.

Modern workflow orchestration — turn any Python function into a scheduled, observable, retriable data or AI pipeline with zero boilerplate.

Framework-agnostic MLOps framework — build portable, production-ready ML pipelines that run anywhere.

Git-like version control for ML datasets and pipelines — track experiments, reproduce results, and collaborate on data science projects.

Apache-2.0 LLM testing framework — pytest-style unit tests for RAG pipelines, chatbots, and LLM outputs.

LLM eval + red-teaming framework — test prompts and models against custom assertions, compare providers, and catch regressions in CI.

Open-source LLM evaluation and observability platform — trace, test, and monitor LLM apps end-to-end.