> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://www.comet.com/docs/opik/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://www.comet.com/docs/opik/_mcp/server.

# Few-Shot Bayesian Optimizer

> Learn how to use the Few-Shot Bayesian Optimizer to find optimal few-shot examples for your chat-based prompts using Bayesian optimization techniques.

The FewShotBayesianOptimizer is a sophisticated prompt optimization tool adds relevant examples from
your sample questions to the system prompt using Bayesian optimization techniques.

The `FewShotBayesianOptimizer` is a strong choice when your primary goal is to find the optimal number and
combination of few-shot examples (demonstrations) to accompany your main instruction prompt,
particularly for **chat models**. If your task performance heavily relies on the quality and relevance of in-context examples, this optimizer is ideal.

## How It Works

The `FewShotBayesianOptimizer` uses Bayesian optimization to find the optimal set and number of
few-shot examples to include with your base instruction prompt for chat models. It Uses
[Optuna](https://optuna.org/), a hyperparameter optimization framework, to guide the search for the
optimal set and number of few-shot examples.

![FewShot Bayesian Optimizer](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/opik.docs.buildwithfern.com/345b3413b4fd2c3109d59fd857596e971083192ddbe3d5e81affd60a2c427490/img/agent_optimization/fewshot_bayesian_optimizer.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260912%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260912T000450Z&X-Amz-Expires=604800&X-Amz-Signature=67b16a46c0e8e44bd8d48e18241edc274cdeeb4df05a0912a7912d756aefcfad&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

## Quickstart

You can use the `FewShotBayesianOptimizer` to optimize a prompt by following these steps:

```python maxLines=1000
from opik_optimizer import FewShotBayesianOptimizer
from opik.evaluation.metrics import LevenshteinRatio
from opik_optimizer import datasets, ChatPrompt

# Initialize optimizer
optimizer = FewShotBayesianOptimizer(
    model="openai/gpt-4",
    model_parameters={
        "temperature": 0.1,
        "max_tokens": 5000
    },
)

# Prepare dataset
dataset = datasets.hotpot(count=300)

# Define metric and prompt (see docs for more options)
def levenshtein_ratio(dataset_item, llm_output):
    return LevenshteinRatio().score(reference=dataset_item["answer"], output=llm_output)

prompt = ChatPrompt(
    messages=[
        {"role": "system", "content": "Provide an answer to the question."},
        {"role": "user", "content": "{question}"}
    ]
)

# Run optimization
results = optimizer.optimize_prompt(
    prompt=prompt,
    dataset=dataset,
    metric=levenshtein_ratio,
    n_samples=100
)

# Access results
results.display()
```

## Configuration Options

### Optimizer parameters

The optimizer has the following parameters:

**`model`** `str` — default: openai/gpt-5-nano

LiteLLM model name for optimizer's internal reasoning (generating few-shot templates)

---

**`model_parameters`** `dict[str, typing.Any] | None`

Optional dict of LiteLLM parameters for optimizer's internal LLM calls. Common params: temperature, max\_tokens, max\_completion\_tokens, top\_p.

---

**`min_examples`** `int` — default: 2

Minimum number of examples to include in the prompt

---

**`max_examples`** `int` — default: 8

Maximum number of examples to include in the prompt

---

**`n_threads`** `int` — default: 8

Number of threads for parallel evaluation

---

**`verbose`** `int` — default: 1

Controls internal logging/progress bars (0=off, 1=on)

---

**`seed`** `int` — default: 42

Random seed for reproducibility

---

### `optimize_prompt` parameters

The `optimize_prompt` method has the following parameters:

**`prompt`** `ChatPrompt`

The prompt to optimize

---

**`dataset`** `Dataset`

Opik Dataset to optimize on

---

**`metric`** `Callable`

Metric function to evaluate on

---

**`experiment_config`** `dict | None`

Optional configuration for the experiment, useful to log additional metadata

---

**`n_samples`** `int | float | str | None`

Number of dataset items to use per evaluation. Use counts (e.g., `50`), fractions (e.g., `0.1`), percentages (e.g., "10%"), or "all"/"full"/None for the full dataset.

---

**`n_samples_minibatch`** `int | None`

Optional number of samples for inner-loop minibatches (defaults to n\_samples).

---

**`n_samples_strategy`** `str | None`

Sampling strategy for subsampling (default: "random\_sorted").

---

**`auto_continue`** `bool` — default: False

Whether to auto-continue optimization

---

**`agent_class`** `type[opik_optimizer.optimizable_agent.OptimizableAgent] | None`

Optional agent class to use

---

**`project_name`** `str` — default: Optimization

Opik project name for logging traces (default: "Optimization")

---

**`max_trials`** `int` — default: 10

Number of trials for Bayesian Optimization (default: 10)

---

**`args`** `Any`

---

**`kwargs`** `Any`

---

### Model Support

There are two models to consider when using the `FewShotBayesianOptimizer`:

* `FewShotBayesianOptimizer.model`: The model used to generate the few-shot template and placeholder.
* `ChatPrompt.model`: The model used to evaluate the prompt.

The `model` parameter accepts any LiteLLM-supported model string (e.g., `"gpt-4o"`, `"azure/gpt-4"`,
`"anthropic/claude-3-opus"`, `"gemini/gemini-1.5-pro"`). You can also pass in extra model parameters
using the `model_parameters` parameter:

```python
optimizer = FewShotBayesianOptimizer(
    model="anthropic/claude-3-opus-20240229",
    model_parameters={
        "temperature": 0.7,
        "max_tokens": 4096
    }
)
```

## Next Steps

1. Explore specific [Optimizers](/development/optimization-runs/algorithms/overview) for algorithm details.
2. Refer to the [FAQ](/development/optimization-runs/faq) for common questions and troubleshooting.
3. Refer to the [API Reference](/development/optimization-runs/advanced/api_reference) for detailed configuration options.