Fireworks AI

Fireworks AI is a production-focused LLM inference platform that specializes in making open-source models reliable enough for real applications. Where other providers focus on raw speed, Fireworks emphasizes structured output, function calling, and JSON mode for open models — capabilities typically only available with proprietary APIs.

• Reliable structured output — enforce JSON schemas, grammar-constrained decoding, and function calling on open models like Llama and Mixtral
• Production inference — optimized serving with speculative decoding, continuous batching, and automatic model routing for cost/speed tradeoffs
• Custom model deployment — upload fine-tuned models and serve them on Fireworks' optimized infrastructure with auto-scaling
• Embedding models — serve open embedding models (Nomic, BGE) at high throughput for RAG pipelines
• OpenAI-compatible API — drop-in replacement for the OpenAI SDK, including tool use and JSON mode parameters
• FireFunction — Fireworks' own function-calling optimized model that matches GPT-4 tool-use accuracy with open-model pricing

Backend engineers and AI teams building production applications use Fireworks when they need open-model economics with proprietary-model reliability. It is particularly valuable for teams building agentic systems where reliable function calling and structured output are non-negotiable.

Fireworks also offers a model composition feature that lets you route requests between different models based on complexity, optimizing for cost on simple queries and quality on hard ones.

1. Create an account — sign up at fireworks.ai and get your API key from the API keys page.
2. Use with the OpenAI SDK:

   from openai import OpenAI
   
   client = OpenAI(
       api_key='your-fireworks-key',
       base_url='https://api.fireworks.ai/inference/v1'
   )
   
   response = client.chat.completions.create(
       model='accounts/fireworks/models/llama-v3p1-70b-instruct',
       messages=[{'role': 'user', 'content': 'Hello!'}]
   )

3. Use JSON mode — enforce structured output:

   response = client.chat.completions.create(
       model='accounts/fireworks/models/llama-v3p1-70b-instruct',
       response_format={'type': 'json_object'},
       messages=[{'role': 'user', 'content': 'List 3 colors as JSON'}]
   )

4. Use function calling — define tools and let the model call them, just like the OpenAI function calling API.
5. Deploy a custom model — upload your fine-tuned LoRA or full model via the dashboard and deploy it on optimized infrastructure.

Pricing: Llama 3.1 8B: $0.20/M tokens. Llama 3.1 70B: $0.90/M tokens. Mixtral 8x22B: $1.20/M tokens. Embeddings from $0.008/M tokens. Free credits on signup. Full pricing details.