LiteLLM

LiteLLM provides a unified OpenAI-compatible interface to call 100+ LLM providers — including OpenAI, Anthropic, Google, Azure, Bedrock, and self-hosted models — without changing your application code. Switch providers by changing a single model string.

Key use cases include:

• Provider abstraction — call any LLM through one consistent API
• Fallback chains — automatically retry on a different provider if one fails
• Cost tracking — monitor spend across all providers in a single dashboard
• Load balancing — distribute requests across multiple deployments
• Proxy server — provide a single OpenAI-compatible endpoint for your entire team
• Budget management — set per-user and per-team spending limits

LiteLLM is used by engineering teams managing multiple LLM providers who want to avoid vendor lock-in and simplify their integration layer. The proxy server is particularly popular for teams that want centralized API key management and cost controls.

It works both as a Python SDK (drop-in replacement for the OpenAI client) and as a standalone proxy server that any application can call.

1. Install LiteLLM:

   pip install litellm

2. Set API keys for your providers:

   export OPENAI_API_KEY='sk-...'
   export ANTHROPIC_API_KEY='sk-ant-...'
   export GEMINI_API_KEY='AIza...'

3. Use as a Python SDK:

   from litellm import completion
   response = completion(model='gpt-4o', messages=[{'role': 'user', 'content': 'Hi'}])
   response = completion(model='claude-sonnet-4-20250514', messages=[{'role': 'user', 'content': 'Hi'}])

4. Or run as a proxy server:

   litellm --model gpt-4o --port 4000
   # Now call http://localhost:4000 like the OpenAI API

Pricing: LiteLLM is free and open source (MIT). An optional hosted proxy (LiteLLM Enterprise) is available for team management features. You pay only for the underlying LLM API calls.