> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://www.comet.com/docs/opik/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://www.comet.com/docs/opik/_mcp/server.

# Optimize agents

> Connect the Agent Optimizer to Agent Frameworks.

The Opik Agent Optimizer can optimize both simple prompts and complex agent workflows. For most use cases, you can optimize prompts directly using `ChatPrompt`. When you need multi-prompt workflows, agent orchestration, or custom execution logic, you'll use `OptimizableAgent` to create a custom agent class.

## When to use OptimizableAgent vs ChatPrompt

**Use `ChatPrompt` directly** (default approach):

* Single-prompt optimization - optimizing one prompt template
* Most common use case
* No custom execution logic needed

**Use `OptimizableAgent`** when you need:

* Multi-prompt workflows - orchestrating multiple prompts in sequence
* Agent framework integration - connecting to ADK, LangGraph, CrewAI, etc.
* Custom execution logic - special tool handling, async workflows, etc.

Optimizers work seamlessly with both approaches. The optimizer calls your agent's `invoke_agent()` method repeatedly during optimization, passing different prompt candidates to evaluate.

## Single-prompt optimization

For most optimization tasks, you can use `ChatPrompt` directly without creating a custom agent. The optimizer uses a default LiteLLM-based agent under the hood.

```python
from opik_optimizer import ChatPrompt, MetaPromptOptimizer
from opik.evaluation.metrics import LevenshteinRatio
from opik_optimizer.datasets import hotpot

dataset = hotpot(count=300)

def levenshtein_ratio(dataset_item, llm_output):
    return LevenshteinRatio().score(
        reference=dataset_item["answer"], 
        output=llm_output
    )

prompt = ChatPrompt(
    system="You are a helpful assistant.",
    user="{question}",
    model="openai/gpt-4o-mini"
)

optimizer = MetaPromptOptimizer(model="openai/gpt-4o")
result = optimizer.optimize_prompt(
    prompt=prompt,
    dataset=dataset,
    metric=levenshtein_ratio,
    max_trials=5,
    n_samples=50
)

result.display()
```

### Custom agent for framework integration

When integrating with specific agent frameworks (Google ADK, LangGraph, CrewAI, etc.), you'll create a custom `OptimizableAgent` subclass. This allows the optimizer to work with your framework's execution model.

Here's an example for Google ADK:

```python
from typing import Any, TYPE_CHECKING
from opik_optimizer import OptimizableAgent

if TYPE_CHECKING:
    from opik_optimizer.api_objects import chat_prompt

class ADKAgent(OptimizableAgent):
    project_name = "adk-agent"

    def invoke_agent(
        self,
        prompts: dict[str, chat_prompt.ChatPrompt],
        dataset_item: dict[str, Any],
        allow_tool_use: bool = False,
        seed: int | None = None,
    ) -> str:
        # Single-prompt agents extract the prompt from the dict
        if len(prompts) > 1:
            raise ValueError("ADKAgent only supports single-prompt optimization.")
        
        prompt = list(prompts.values())[0]
        messages = prompt.get_messages(dataset_item)
        
        # Your framework-specific execution logic here
        # ... create ADK agent, run it, return response ...
        
        return response
```

The key points:

* Extract the single prompt from `prompts` dict: `prompt = list(prompts.values())[0]`
* Get formatted messages: `messages = prompt.get_messages(dataset_item)`
* Execute using your framework and return the response string

See `sdks/opik_optimizer/scripts/llm_frameworks/` for working examples of framework integrations (ADK, LangGraph, CrewAI, etc.). Each script doubles as both documentation and regression tests.

## Multi-prompt optimization

For multi-step agent workflows, you **must** use `OptimizableAgent` because `ChatPrompt` only handles a single prompt. Multi-prompt optimization allows you to optimize multiple prompts that work together in a pipeline.

### When to use multi-prompt optimization

* Sequential reasoning workflows (analyze → respond)
* Multi-hop retrieval pipelines
* Agent orchestration with multiple steps
* Any workflow where one prompt's output feeds into another

### Implementing a multi-prompt agent

Here's a simple example of a two-step workflow that analyzes input and then generates a response:

```python
from typing import Any
from opik_optimizer import ChatPrompt, OptimizableAgent
from openai import OpenAI

class AnalyzeRespondAgent(OptimizableAgent):
    """Two-step agent: analyze input, then respond based on analysis."""
    
    def __init__(self, model: str = "gpt-4o-mini"):
        super().__init__()
        self.model = model
        self.client = OpenAI()

    def invoke_agent(
        self,
        prompts: dict[str, ChatPrompt],
        dataset_item: dict[str, Any],
        allow_tool_use: bool = False,
        seed: int | None = None,
    ) -> str:
        # Step 1: Analyze the input
        analyze_prompt = prompts["analyze"]
        analyze_messages = analyze_prompt.get_messages(dataset_item)
        
        analyze_response = self.client.chat.completions.create(
            model=self.model,
            messages=analyze_messages,
            seed=seed,
        )
        analysis = analyze_response.choices[0].message.content

        # Step 2: Generate response based on analysis
        respond_prompt = prompts["respond"]
        # Pass analysis result to the respond prompt
        respond_context = {**dataset_item, "analysis": analysis}
        respond_messages = respond_prompt.get_messages(respond_context)
        
        respond_response = self.client.chat.completions.create(
            model=self.model,
            messages=respond_messages,
            seed=seed,
        )
        
        return respond_response.choices[0].message.content
```

### Using the multi-prompt agent

When optimizing, pass a dictionary of prompts instead of a single prompt:

```python
from opik_optimizer import ChatPrompt, MetaPromptOptimizer
from opik.evaluation.metrics import LevenshteinRatio

# Define both prompts in the workflow
prompts = {
    "analyze": ChatPrompt(
        system="You are an analysis assistant. Extract key information from the input.",
        user="{text}",
        model="gpt-4o-mini"
    ),
    "respond": ChatPrompt(
        system="You are a response assistant. Generate a helpful response based on the analysis.",
        user="Analysis: {analysis}\n\nOriginal question: {text}",
        model="gpt-4o-mini"
    ),
}

optimizer = MetaPromptOptimizer(model="openai/gpt-4o")
result = optimizer.optimize_prompt(
    prompt=prompts,  # Pass dict of prompts
    agent_class=AnalyzeRespondAgent,  # Use your custom agent
    dataset=dataset,
    metric=levenshtein_ratio,
    max_trials=5,
    n_samples=50
)

result.display()
```

The optimizer will optimize both prompts in the dictionary, trying different combinations to improve performance.

The prompts dict keys (like "analyze" and "respond") are used to identify which prompt to optimize. The optimizer can optimize all prompts or specific ones based on the `optimize_prompt` parameter.

## Key implementation details

### invoke\_agent() method signature

All `OptimizableAgent` subclasses must implement `invoke_agent()`:

```python
def invoke_agent(
    self,
    prompts: dict[str, ChatPrompt],
    dataset_item: dict[str, Any],
    allow_tool_use: bool = False,
    seed: int | None = None,
) -> str:
    # Your implementation here
    return response_string
```

**Parameters:**

* `prompts`: Dictionary mapping prompt names to `ChatPrompt` objects
* `dataset_item`: Dataset row used to format prompt messages
* `allow_tool_use`: Whether tools may be executed (for tool-calling prompts)
* `seed`: Optional random seed for reproducibility

**Returns:** A single string output that will be scored by your metric function

### Extracting messages from prompts

Use `ChatPrompt.get_messages()` to format the prompt with dataset values:

```python
messages = prompt.get_messages(dataset_item)
# Returns list of message dicts: [{"role": "system", "content": "..."}, ...]
```

For multi-prompt workflows, pass additional context when calling `get_messages()`:

```python
# Pass intermediate results to subsequent prompts
context = {**dataset_item, "intermediate_result": some_value}
messages = prompt.get_messages(context)
```

### Best practices

* **Error handling**: Return meaningful error messages if execution fails
* **Model parameters**: Respect `prompt.model` and `prompt.model_kwargs` for consistency
* **Reproducibility**: Use the `seed` parameter when making LLM calls
* **Opik tracing**: The base class handles tracing automatically, but you can add custom metadata via `self.trace_metadata`

## Complete examples

### Single-prompt with ChatPrompt (default)

```python
from opik_optimizer import ChatPrompt, EvolutionaryOptimizer
from opik_optimizer.datasets import hotpot
from opik.evaluation.metrics import LevenshteinRatio

dataset = hotpot(count=300)

def metric(dataset_item, llm_output):
    return LevenshteinRatio().score(
        reference=dataset_item["answer"], 
        output=llm_output
    )

prompt = ChatPrompt(
    system="You are a helpful assistant.",
    user="{question}",
    model="openai/gpt-4o-mini"
)

optimizer = EvolutionaryOptimizer(
    model="openai/gpt-4o-mini",
    population_size=5,
    num_generations=3
)

result = optimizer.optimize_prompt(
    prompt=prompt,
    dataset=dataset,
    metric=metric,
    n_samples=50
)

result.display()
```

### Multi-prompt workflow

```python
from typing import Any
from opik_optimizer import ChatPrompt, OptimizableAgent, HRPO
from opik.evaluation.metrics import LevenshteinRatio
from opik_optimizer.datasets import hotpot
from openai import OpenAI

class TwoStepAgent(OptimizableAgent):
    def __init__(self, model: str = "gpt-4o-mini"):
        super().__init__()
        self.model = model
        self.client = OpenAI()

    def invoke_agent(
        self,
        prompts: dict[str, ChatPrompt],
        dataset_item: dict[str, Any],
        allow_tool_use: bool = False,
        seed: int | None = None,
    ) -> str:
        # First step
        step1_prompt = prompts["step1"]
        step1_messages = step1_prompt.get_messages(dataset_item)
        step1_response = self.client.chat.completions.create(
            model=self.model,
            messages=step1_messages,
            seed=seed,
        )
        step1_result = step1_response.choices[0].message.content

        # Second step uses result from first step
        step2_prompt = prompts["step2"]
        step2_context = {**dataset_item, "step1_result": step1_result}
        step2_messages = step2_prompt.get_messages(step2_context)
        step2_response = self.client.chat.completions.create(
            model=self.model,
            messages=step2_messages,
            seed=seed,
        )
        
        return step2_response.choices[0].message.content

# Define multi-prompt workflow
prompts = {
    "step1": ChatPrompt(
        system="Analyze the question and identify key information.",
        user="{question}",
        model="gpt-4o-mini"
    ),
    "step2": ChatPrompt(
        system="Answer the question based on the analysis.",
        user="Question: {question}\n\nAnalysis: {step1_result}",
        model="gpt-4o-mini"
    ),
}

dataset = hotpot(count=300)

def metric(dataset_item, llm_output):
    return LevenshteinRatio().score(
        reference=dataset_item["answer"], 
        output=llm_output
    )

optimizer = HRPO(
    model="openai/gpt-4o-mini",
    n_threads=2,
    max_parallel_batches=3
)

result = optimizer.optimize_prompt(
    prompt=prompts,
    agent_class=TwoStepAgent,
    dataset=dataset,
    metric=metric,
    max_trials=5,
    n_samples=50
)

result.display()
```

For advanced multi-prompt examples, see `sdks/opik_optimizer/benchmarks/agents/hotpot_multihop_agent.py` which implements a complex multi-hop retrieval pipeline with Wikipedia search.

## Next steps

* Explore [optimization algorithms](/development/optimization-runs/algorithms/overview) to choose the right optimizer
* Learn about [defining datasets](/development/optimization-runs/optimization/define_datasets) and [metrics](/development/optimization-runs/optimization/define_metrics)
* Check framework-specific examples in `sdks/opik_optimizer/scripts/llm_frameworks/`