New! Write simple unit tests and let Opik debug your agents for you. Here’s how→

open-source llm observability

AI Observability & Evals
For the Agentic Era

Opik logs every step your agent takes, from user interactions to context retrieval and tool calls — with automated eval workflows to find and fix errors across development, testing, and production.

Understand what your agent is doing,
where it’s failing, and how to fix it.

With end-to-end observability and evaluation tooling, Opik lets you confidently scale agents from prototype to production. Comprehensive logs, repeatable test cycles, and straightforward evaluation scores ensure consistent performance and help you build trust with end users and internal stakeholders alike.

Trace & Debug Any Step in Your AI System

Capture, visualize, and understand every action your agent takes.
Collaborate with subject matter experts to annotate and fix underperforming traces.
Automatically produce audit logs for your governance team.

Evaluate Outcomes with LLM-as-a-Judge Metrics

Define what good looks like with a reference dataset or a plain-text assertion, and let Opik surface errors out of thousands of traces.
Evaluate traces from development, testing, or production to compare agent versions ship with confidence.
Score performance with 30+ metrics for answer relevance, context precision, task completion, hallucination, and more.

Monitor Your Agents in Production

Evaluate production traces in real time and get alerted if a user interaction fails your test criteria.
Apply guardrails to proactively block content and policy violations and protect against PII exposure and other compliance risks.
Track token usage and model cost and find where to optimize for efficiency.

Built for developers. Trusted by the world’s largest enterprise teams.

The Opik Difference: Automatically Fix Your Agent’s Codebase

Define plain-text assertions for your desired outcomes in Test Suites, auto-implement fixes with the Ollie coding harness, and test run your entire agent in Agent Playground.

Test Suites & Assertions: Define Unit Tests

Define rules for what your agent should and shouldn’t do, and get clear pass/fail results. Set global rules that every test case must pass, plus item-level assertions for specific scenarios. No need to create individual eval metrics, reference datasets, or run one-off evals.

Ollie: Write Fixes Directly to Your Codebase

Opik’s powerful coding assistant analyzes your traces, suggests fixes, and implements them in your development code — with built-in version control and regression testing. With every fix, Ollie writes a new test case to ensure the same issue won’t slip through again.

Agent Playground: Test Agents End-to-End

Run your entire agent in Opik to understand how changes to your configuration of models, prompts, and parameters affect the system as a whole. Track and version sets of prompts and parameters and deploy successful versions. Give stakeholders outside your dev team access to test and experiment safely.

Prompt Optimizer: Maximize Agent Performance

Choose from seven advanced prompt optimization algorithms to achieve more precise and consistent results throughout your agent, from orchestration and tool calling steps to model parameters and user interactions.

Open Source & Ready to Run

Opik is a true open-source project, and its core AI observability and evaluation feature set is included free in the source code. You can download the code from GitHub and run it locally, with a highly scalable and industry-compliant version ready for enterprise teams.

Iterate Across Your Agent
Development Lifecycle

Opik helps analyze the quality of LLM responses at every step of the app development lifecycle so you can debug and optimize with confidence.

Understand Cause & Effect in Complex Agentic Systems

With multiple components influencing model behavior and countless outputs generated during development, manual review and vibe checks don’t cut it.

With Opik, you can log traces and compute scores in the aggregate, and drill down to individual prompts and responses that need attention.

Opik LLM lifecycle: three stages in a loop. Development: iterate on prompts and context retrieval for accurate LLM outputs. Unit Testing: verify performance across pipelines, prompts, and models. Production: validate on unseen data and generate datasets for the next cycle.

Opik LLM lifecycle: three stages in a loop. Development: iterate on prompts and context retrieval for accurate LLM outputs. Unit Testing: verify performance across pipelines, prompts, and models. Production: validate on unseen data and generate datasets for the next cycle.

Try Opik Free

You don’t need a credit card to sign up, and your Comet account comes with a generous free tier you can actually use — for as long as you like.

Create Free Account