LiteLLM

LiteLLM provides a single OpenAI-compatible Python SDK and proxy server for 100+ LLM providers. Switch between models and providers by changing one string — no SDK swaps, no API re-integration, no prompt reformatting.

Key features

• 100+ providers — OpenAI, Anthropic, Google Gemini, Cohere, Mistral, Groq, Together AI, Ollama, Azure, AWS Bedrock, and more
• Proxy server — drop-in OpenAI-compatible HTTP proxy with load balancing, rate limiting, and cost tracking
• Consistent output — normalizes all provider responses to the OpenAI ChatCompletion schema
• Fallbacks — automatic failover to backup models when primary fails
• Cost tracking — per-request and per-user cost logging with budget limits
• Async — full async/streaming support with acompletion

Quick start

npx ai-supply add litellm-llm-gateway

# Or install directly
pip install litellm

import litellm

# Call any provider with identical syntax
response = litellm.completion(
    model="anthropic/claude-opus-4-5",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)

# Switch to Gemini — same code
response = litellm.completion(
    model="gemini/gemini-1.5-pro",
    messages=[{"role": "user", "content": "Hello!"}]
)

# Or run as a proxy
litellm --model anthropic/claude-opus-4-5 --port 8000
# Now any OpenAI client can hit http://localhost:8000

Curated mirror of the open-source LiteLLM project (MIT). Install upstream from the repository.