Langfuse

What is Langfuse?

Langfuse is an open source LLM engineering platform that helps teams build, monitor, evaluate, and debug AI applications. As developers move from simple prototypes to production-ready generative AI tools, the complexity of application logic increases. Standard debugging and logging techniques often fall short when dealing with non-deterministic outputs from large language models (LLMs). An application might combine several operations—such as retrieving context from a vector database, generating embeddings, and prompting multiple LLMs in a sequence—before returning a final answer to the user.

To solve this visibility problem, Langfuse offers robust tracing capabilities. It provides observability features to trace LLM calls, retrieval steps, and agent actions, allowing developers to debug complex logs. By inspecting user sessions, developers can visualize the exact sequence of events, track latency, monitor token usage, and identify exactly where an error or hallucination originated. This comprehensive observability ensures that teams can quickly address issues and optimize the cost and performance of their AI applications.

Centralized Prompt Management and Evaluation

As AI projects scale, managing prompt variations across a team becomes a significant challenge. Hardcoding prompts into the application's source code can lead to scattered files, making it difficult to test new versions or roll back problematic changes. Langfuse includes a Prompt Management system that allows teams to centrally manage, version control, and iterate on prompts. This centralized hub enables non-engineers to collaborate on prompt design and allows developers to fetch the latest configurations without needing to redeploy the underlying application code.

Furthermore, changing a prompt or swapping an underlying model requires rigorous testing to ensure output quality does not degrade. The platform supports various evaluation workflows, including LLM-as-a-judge, code evaluators, and user feedback collection. Teams can curate test sets and benchmarks within Langfuse, running structured experiments to score outputs systematically. By measuring metrics such as relevance, conciseness, or factual accuracy, developers can deploy updates with confidence.

Running Langfuse Locally

For teams that want full control over their data, Langfuse is designed to be easily self-hosted. It integrates natively with many popular frameworks—such as LangChain, LlamaIndex, and OpenAI's SDK—making instrumentation straightforward. Before deploying to a production cluster via Kubernetes or a virtual machine, developers can run Langfuse locally in minutes using Docker Compose.

To evaluate the platform on your own machine, you can clone the canonical repository and launch the application using the official deployment commands:

Once the Docker containers have started, you will have access to the local dashboard. From there, you can create a project, generate API keys, and begin instrumenting your application to trace LLM interactions immediately.