> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://www.comet.com/docs/opik/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://www.comet.com/docs/opik/_mcp/server.

# GEPA Optimizer

> Use the external GEPA package through Opik's `GepaOptimizer` to optimize a single system prompt for single-turn tasks with a reflection model.

`GepaOptimizer` wraps the external [GEPA](https://github.com/gepa-ai/gepa) package to optimize a
single system prompt for single-turn tasks. It maps Opik datasets and metrics into GEPA’s expected
format, runs GEPA’s optimization using a task model and a reflection model, and returns a standard
`OptimizationResult` compatible with the Opik SDK.

`GepaOptimizer` is ideal when you have a single-turn task (one user input → one model
response) and you want to optimize the system prompt using a reflection-driven search.

## How it works

The GEPA optimizer companies two key approaches to optimize agents:

1. **Reflection**: The optimizer uses the outcomes from evaluations to improve the prompts.
2. **Evolution**: The optimizer uses an evolutionary algorithm to explore the space of prompts.

You can learn more about the algorithm in the [GEPA paper](https://arxiv.org/abs/2507.19457) but in
short, the optimizer will:

![GEPA Optimizer](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/opik.docs.buildwithfern.com/dbd510a3af98454deb7d178897c244838a025c4b3db5ac88170c9762d62a9420/img/agent_optimization/gepa_optimizer.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260911%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260911T194708Z&X-Amz-Expires=604800&X-Amz-Signature=9e24e1c38902f798ae8bd588e4132b1b1cb7dad1eb91c29bba62c799dc4c6dee&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

## Quickstart

```python
"""
Optimize a simple system prompt on the tiny_test dataset.
Requires: pip install gepa, and a valid OPENAI_API_KEY for LiteLLM-backed models.
"""
from typing import Any, Dict

from opik.evaluation.metrics import LevenshteinRatio
from opik.evaluation.metrics.score_result import ScoreResult

from opik_optimizer import ChatPrompt, datasets
from opik_optimizer.gepa_optimizer import GepaOptimizer

def levenshtein_ratio(dataset_item: Dict[str, Any], llm_output: str) -> ScoreResult:
    return LevenshteinRatio().score(reference=dataset_item["label"], output=llm_output)

dataset = datasets.tiny_test()

prompt = ChatPrompt(
    system="You are a helpful assistant. Answer concisely with the exact answer.",
    user="{text}",
)

optimizer = GepaOptimizer(
    model="openai/gpt-4o-mini",
    n_threads=6,
    model_parameters={"temperature": 0.2, "max_tokens": 200},
)

result = optimizer.optimize_prompt(
    prompt=prompt,
    dataset=dataset,
    metric=levenshtein_ratio,
    max_trials=12,
    reflection_minibatch_size=2,
    n_samples=5,
)

result.display()
```

### Determinism and tool usage

* GEPA’s seed is forwarded directly to the underlying `gepa.optimize` call, but any non-determinism in your prompt (tool calls, non-zero temperature, external APIs) will still introduce variance. To test seeding in isolation, disable tools or substitute cached responses.
* GEPA emits its own baseline evaluation inside the optimization loop. You’ll see one baseline score from Opik’s wrapper and another from GEPA before the first trial; this is expected and does not double-charge the metric budget.
* Reflection only triggers after GEPA accepts at least `reflection_minibatch_size` unique prompts. If the minibatch is larger than the trial budget, the optimizer logs a warning and skips reflection.
* Reflection calls use `GepaOptimizer.model`, are traced as Opik spans, and count toward the run's reported LLM cost. They are **not** metric calls, so `max_trials` does not bound them — a run typically makes many more reflection calls than trials, because an iteration whose candidate loses on the mini-batch costs only `2 * reflection_minibatch_size` metric calls. The run reports the total as `reflection_call_count` in the result details; set `max_reflection_calls` to cap it.
* GEPA supports **tool use during evaluation** (`allow_tool_use=True`) but does **not** support `optimize_tools=True` yet. Tool-description optimization requests are currently degraded/blocked until the adapter supports it.

### GEPA scores vs. Opik scores

* The **GEPA Score** column reflects the aggregate score GEPA computes on its train/validation split when deciding which candidates stay on the Pareto front. It is useful for understanding how GEPA’s evolutionary search ranks prompts.
* The **Opik Score** column is a fresh evaluation performed through Opik’s metric pipeline on the same dataset (respecting `n_samples`). This is the score you should use when comparing against your baseline or other optimizers.
* Because the GEPA score is based on GEPA’s internal aggregation, it can diverge from the Opik score for the same prompt. This is expected—treat the GEPA score as a hint about why GEPA kept or discarded a candidate, and rely on the Opik score for final comparisons.

### `skip_perfect_score`

* When `skip_perfect_score=True`, GEPA immediately ignores any candidate whose GEPA score meets or exceeds the `perfect_score` threshold (default `1.0`). This keeps the search moving toward imperfect prompts instead of spending budget refining already perfect ones.
* Set `skip_perfect_score=False` if your metric tops out below `1.0`, or if you still want to see how GEPA mutates a perfect-scoring prompt—for example, when you care about ties being broken by Opik’s rescoring step rather than GEPA’s aggregate.

## Configuration Options

### Optimizer parameters

The optimizer has the following parameters:

**`model`** `str` — default: openai/gpt-5-nano

LiteLLM model name for the optimization algorithm

---

**`model_parameters`** `dict[str, typing.Any] | None`

Optional dict of LiteLLM parameters for optimizer's internal LLM calls. Common params: temperature, max\_tokens, max\_completion\_tokens, top\_p.

---

**`n_threads`** `int` — default: 6

Number of parallel threads for evaluation

---

**`verbose`** `int` — default: 1

Controls internal logging/progress bars (0=off, 1=on)

---

**`seed`** `int` — default: 42

Random seed for reproducibility

---

### `optimize_prompt` parameters

The `optimize_prompt` method has the following parameters:

**`prompt`** `ChatPrompt`

The prompt to optimize

---

**`dataset`** `Dataset`

Opik Dataset to optimize on

---

**`metric`** `Callable`

Metric function to evaluate on

---

**`experiment_config`** `dict | None`

Optional configuration for the experiment

---

**`n_samples`** `int | float | str | None`

Number of dataset items to use per evaluation. Use counts (e.g., `50`), fractions (e.g., `0.1`), percentages (e.g., "10%"), or "all"/"full"/None for the full dataset.

---

**`n_samples_minibatch`** `int | None`

Optional number of samples for inner-loop minibatches (defaults to n\_samples).

---

**`n_samples_strategy`** `str | None`

Sampling strategy for subsampling (default: "random\_sorted").

---

**`auto_continue`** `bool` — default: False

Whether to auto-continue optimization

---

**`agent_class`** `type[opik_optimizer.optimizable_agent.OptimizableAgent] | None`

Optional agent class to use

---

**`project_name`** `str` — default: Optimization

---

**`max_trials`** `int` — default: 10

Maximum number of different prompts to test (default: 10)

---

**`reflection_minibatch_size`** `int` — default: 3

Size of reflection minibatches (default: 3)

---

**`max_reflection_calls`** `int` — default: 0

Cap on reflection-LLM calls — the calls GEPA makes to propose new candidates, which are not metric calls and are billed on top of them. Defaults to `0`, meaning no cap: GEPA reflects as often as its search requires. Set a positive value to bound reflection spend; the run then stops once the cap is reached and reports `finish_reason="reflection_budget"` (a run that spent its full metric budget at the same time reports `max_trials`, since the trials are the reason it ended). Reflection calls scale with engine iterations rather than trials, so pick this from the run's reported `reflection_call_count`, not from `max_trials`.

---

**`candidate_selection_strategy`** `str` — default: pareto

Strategy for candidate selection (choose from "pareto", "current\_best", or "epsilon\_greedy"; default: "pareto")

---

**`skip_perfect_score`** `bool` — default: True

Skip candidates with perfect scores (default: True)

---

**`perfect_score`** `float` — default: 1.0

Score considered perfect (default: 1.0)

---

**`use_merge`** `bool` — default: False

Enable merge operations (default: False)

---

**`max_merge_invocations`** `int` — default: 5

Maximum merge invocations (default: 5)

---

**`run_dir`** `str | None`

Directory for run outputs (default: None)

---

**`track_best_outputs`** `bool` — default: False

Track best outputs during optimization (default: False)

---

**`display_progress_bar`** `bool` — default: False

Display progress bar (default: False)

---

**`seed`** `int` — default: 42

Random seed for reproducibility (default: 42)

---

**`raise_on_exception`** `bool` — default: True

Raise exceptions instead of continuing (default: True)

---

**`kwargs`** `Any`

---

### Model Support

GEPA coordinates two model contexts:

* `GepaOptimizer.model`: LiteLLM model string the optimizer uses for internal reasoning (reflection, mutation prompts, etc.).
* `ChatPrompt.model`: The model evaluated against your dataset—this should match what you run in production.

Set `model` to any LiteLLM-supported provider (e.g., `"gpt-4o"`, `"azure/gpt-4"`, `"anthropic/claude-3-opus"`, `"gemini/gemini-1.5-pro"`) and pass extra parameters via `model_parameters` when you need to tune temperature, max tokens, or other limits:

```python
optimizer = GepaOptimizer(
    model="anthropic/claude-3-opus-20240229",
    model_parameters={
        "temperature": 0.7,
        "max_tokens": 4096
    }
)
```

Reflection is handled internally; there is no separate `reflection_model` argument to set.

## Limitations & tips

* **Instruction-focused**: The current wrapper optimizes the instruction/system portion of your prompt. If you rely heavily on few-shot exemplars, consider pairing GEPA with the Few-Shot Bayesian optimizer or an Evolutionary run.
* **Reflection can misfire**: GEPA’s reflective mutations are only as good as the metric reasons you supply. If `ScoreResult.reason` is vague, the optimizer may reinforce bad behaviors. Invest in descriptive metrics before running GEPA at scale.
* **Cost-aware**: Although GEPA is more sample-efficient than some RL-based methods, reflection and Pareto scoring still consume multiple LLM calls per trial. Start with small `max_trials` and monitor API usage.

## Next Steps

1. Explore specific [Optimizers](/development/optimization-runs/algorithms/overview) for algorithm details.
2. Refer to the [FAQ](/development/optimization-runs/faq) for common questions and troubleshooting.
3. Refer to the [API Reference](/development/optimization-runs/advanced/api_reference) for detailed configuration options.