OpenLLMetry

OpenLLMetry

OpenLLMetry is an open-source observability SDK for LLM applications built on OpenTelemetry standards. Add one line of code to get automatic traces for LangChain, LlamaIndex, OpenAI, Anthropic, Bedrock, VertexAI, and 20+ other frameworks — without vendor lock-in.

Key Features

• One-line instrumentation: Auto-trace any supported LLM framework without modifying application code
• OpenTelemetry native: Sends standard OTLP traces to any backend (Jaeger, Grafana, Datadog, New Relic, Traceloop)
• Prompt & completion capture: Records full prompts, completions, token counts, and model parameters
• Vector DB tracing: Instruments Chroma, Qdrant, Pinecone, Weaviate, and Milvus calls
• 20+ integrations: LangChain, LlamaIndex, Haystack, CrewAI, AutoGen, Transformers, and more
• Privacy controls: Configurable suppression of prompt/completion content for sensitive environments

Quick Start

pip install opentelemetry-sdk traceloop-sdk

from traceloop.sdk import Traceloop

Traceloop.init(app_name="my-llm-app")

# All subsequent OpenAI / LangChain / etc. calls are now traced automatically
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)

Add to ai-supply

npx ai-supply add openllmetry-llm-observability

Curated mirror of the open-source OpenLLMetry (Apache-2.0). Get it from the source.