DevelopmentOptimization runs

Quickstart

View as Markdown

Opik Agent Optimizer Quickstart gives you the fastest path from “hello world” to a successful optimization run. If you already walked through the main Opik Quickstart (tracing + evaluation), this is the next stop—it layers on the opik-optimizer SDK so you can automatically improve prompts and agents. Prefer a UI workflow? Use Optimization Studio instead.

Why Opik Agent Optimizer?

  • Production-grade workflows – reuse the same datasets, metrics, and tracing you already have in Opik.
  • Multiple strategies – swap between MetaPrompt, Hierarchical Reflective Prompt Optimizer (HRPO), Evolutionary, GEPA, and more with one API.
  • Deep analysis – every trial is logged to Opik so you can inspect prompts, tool calls, and failure modes.

Estimated time: ≤10 minutes if you already have Python and an Opik API key configured.

Prerequisites

  • Python 3.10+
  • Opik account
  • Access to an OpenAI-compatible LLM via LiteLLM (OPENAI_API_KEY, ANTHROPIC_API_KEY, etc.)

1. Install and authenticate

pip install --upgrade opik opik-optimizer
opik configure  # paste your API key
export OPIK_PROJECT_NAME="optimization-quickstart"

Setting OPIK_PROJECT_NAME ensures all traces, experiments, and optimization runs are logged to the same project without having to pass project_name to every SDK call.

2. Create a dataset and metric

import opik
from opik.evaluation.metrics import LevenshteinRatio
client = opik.Opik()
dataset = client.get_or_create_dataset(name="agent-opt-quickstart")
dataset.insert([
    {"question": "What is Opik?", "answer": "Opik is an LLM observability and optimization platform."},
    {"question": "How do I reduce hallucinations?", "answer": "Use evaluations and prompt optimization to enforce grounding."},
])
def answer_quality(item, output):
    metric = LevenshteinRatio()
    return metric.score(reference=item["answer"], output=output)

3. Run the optimizer

from opik_optimizer import MetaPromptOptimizer, ChatPrompt
prompt = ChatPrompt(
    messages=[
        {"role": "system", "content": "You are a precise assistant."},
        {"role": "user", "content": "{question}"},
    ],
    model="openai/gpt-5-nano"  # The model your prompt runs on
)
optimizer = MetaPromptOptimizer(model="openai/gpt-5-nano")  # The model that improves your prompt
result = optimizer.optimize_prompt(
    prompt=prompt,
    dataset=dataset,
    metric=answer_quality,
    max_trials=3,
    n_samples=2,
)
result.display()

Using a different LLM provider? The optimizer supports OpenAI, Anthropic, Gemini, Azure, Ollama, and 100+ other providers via LiteLLM. See the Configure LLM Providers guide for setup instructions.

4. Inspect results

  • Run opik dashboard or open https://www.comet.com/opik.
  • In the left nav, go to Evaluation → Optimization runs, then select your latest run.
  • Review the optimization-progress chart, trial table, and per-trial traces to decide whether to ship the new prompt.

Common first issues

Import ChatPrompt from opik_optimizer and wrap your messages list before passing it to any optimizer.

Re-run opik configure and confirm the account has Agent Optimizer access. If you changed machines, copy the ~/.opik/config file or re-enter the key.

Ensure provider keys (e.g., OPENAI_API_KEY) are exported in the same shell running the script, and verify the model you selected is enabled for that key.

Next steps