Langfuse

Langfuse is an open-source LLM observability and tracing platform that captures every prompt, completion, latency, cost, and error across your LLM application. It gives you full visibility into what your AI application is doing in production.

Key use cases include:

• Tracing — capture the full execution trace of every LLM call, tool invocation, and retrieval step
• Cost monitoring — track token usage and spend across providers and models
• Quality evaluation — run LLM-as-judge evaluations and human annotation workflows
• Prompt management — version and deploy prompts with A/B testing
• Dataset management — build evaluation datasets from production traces
• Debugging — quickly identify why specific requests failed or produced poor results

Langfuse is used by engineering teams running LLM applications in production who need to monitor quality, control costs, and debug issues. It integrates with LangChain, LlamaIndex, OpenAI, and every major framework through decorators, callbacks, or the REST API.

Available as Langfuse Cloud (hosted) or self-hosted via Docker for full data control.

1. Sign up at cloud.langfuse.com or self-host with Docker.
2. Install the SDK:

   pip install langfuse

3. Set your keys:

   export LANGFUSE_PUBLIC_KEY='pk-...'
   export LANGFUSE_SECRET_KEY='sk-...'
   export LANGFUSE_HOST='https://cloud.langfuse.com'

4. Add tracing with the @observe() decorator:

   from langfuse.decorators import observe
   
   @observe()
   def my_llm_function(query):
       # Your LLM call here
       return response

5. View traces and analytics in the Langfuse dashboard.

Pricing: Free tier with 50K observations/month. Pro plan at $59/month. Self-hosting is free (MIT license). See langfuse.com/pricing.